[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:13:25 dist-test-slave-7nm7 krb5kdc[26557](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:13:25 dist-test-slave-7nm7 krb5kdc[26557](info): set up 2 sockets
Aug 06 01:13:25 dist-test-slave-7nm7 krb5kdc[26557](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 01:13:27.112394 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.625s user 0.001s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260806 01:13:27.259613 26546 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.147s user 0.000s sys 0.008s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260806 01:13:27.399747 26546 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.140s user 0.002s sys 0.005s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260806 01:13:27.532974 26546 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.133s user 0.000s sys 0.008s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/test-user.keytab.
W20260806 01:13:27.779487 26546 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.246s user 0.000s sys 0.009s
Aug 06 01:13:27 dist-test-slave-7nm7 krb5kdc[26557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:13:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:13:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:13:27.890928 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:39087
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:39087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:28.460152 26576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:28.460640 26576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:28.460819 26576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:28.470407 26576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:28.470593 26576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:28.470662 26576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:28.470742 26576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:28.470996 26576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:28.485447 26576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:39087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:39087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:28.487596 26576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:28.489527 26576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:28.512488 26585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:28.512758 26586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:28.512576 26588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:28.514917 26576 server_base.cc:1034] running on GCE node
I20260806 01:13:28.516697 26576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:28.519553 26576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:28.522998 26576 hybrid_clock.cc:648] HybridClock initialized: now 1785978808522857 us; error 128 us; skew 500 ppm
I20260806 01:13:28.523761 26576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:28 dist-test-slave-7nm7 krb5kdc[26557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978808, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:28.534580 26576 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:13:28.538013 26576 webserver.cc:460] Webserver started at http://127.25.236.190:34533/ using document root <none> and password file <none>
I20260806 01:13:28.539572 26576 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:28.539762 26576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:28.540508 26576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:28.546628 26576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "bec2c52fde404f0a97ec90d96ce67ceb"
format_stamp: "Formatted at 2026-08-06 01:13:28 on dist-test-slave-7nm7"
server_key: "1de6700d781412441022ad29bd69fb03"
server_key_iv: "a2330ea7e984c345245d9efdd82592c8"
server_key_version: "encryptionkey@0"
I20260806 01:13:28.547638 26576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "bec2c52fde404f0a97ec90d96ce67ceb"
format_stamp: "Formatted at 2026-08-06 01:13:28 on dist-test-slave-7nm7"
server_key: "1de6700d781412441022ad29bd69fb03"
server_key_iv: "a2330ea7e984c345245d9efdd82592c8"
server_key_version: "encryptionkey@0"
I20260806 01:13:28.558300 26576 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.005s
I20260806 01:13:28.565874 26600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:28.568250 26576 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.003s
I20260806 01:13:28.568648 26576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "bec2c52fde404f0a97ec90d96ce67ceb"
format_stamp: "Formatted at 2026-08-06 01:13:28 on dist-test-slave-7nm7"
server_key: "1de6700d781412441022ad29bd69fb03"
server_key_iv: "a2330ea7e984c345245d9efdd82592c8"
server_key_version: "encryptionkey@0"
I20260806 01:13:28.568994 26576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:28.589702 26576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:28.597507 26576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:28.598030 26576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:28.639484 26576 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:39087
I20260806 01:13:28.639503 26674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:39087 every 8 connection(s)
I20260806 01:13:28.642072 26576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:13:28.646407 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 26576
I20260806 01:13:28.646862 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:13:28.649390 26546 external_mini_cluster.cc:1428] Setting key 37cc5a27523e386e3a0887039743d129
I20260806 01:13:28.649906 26675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:28.668483 26675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb: Bootstrap starting.
I20260806 01:13:28.677192 26675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:28.679428 26675 log.cc:824] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:28.684675 26675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb: No bootstrap required, opened a new log
Aug 06 01:13:28 dist-test-slave-7nm7 krb5kdc[26557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:28.696650 26675 raft_consensus.cc:348] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec2c52fde404f0a97ec90d96ce67ceb" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39087 } }
I20260806 01:13:28.697544 26675 raft_consensus.cc:374] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:28.697757 26675 raft_consensus.cc:729] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bec2c52fde404f0a97ec90d96ce67ceb, State: Initialized, Role: FOLLOWER
I20260806 01:13:28.699344 26675 consensus_queue.cc:260] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec2c52fde404f0a97ec90d96ce67ceb" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39087 } }
I20260806 01:13:28.699857 26675 raft_consensus.cc:388] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:13:28.700278 26675 raft_consensus.cc:482] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:13:28.700592 26675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:28.705049 26675 raft_consensus.cc:504] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec2c52fde404f0a97ec90d96ce67ceb" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39087 } }
I20260806 01:13:28.706375 26675 leader_election.cc:302] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bec2c52fde404f0a97ec90d96ce67ceb; no voters:
I20260806 01:13:28.707499 26675 leader_election.cc:288] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:13:28.707777 26683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:28.709851 26683 raft_consensus.cc:686] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [term 1 LEADER]: Becoming Leader. State: Replica: bec2c52fde404f0a97ec90d96ce67ceb, State: Running, Role: LEADER
I20260806 01:13:28.710268 26677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:28.655578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37106 (local address 127.25.236.190:39087)
0806 01:13:28.657221 (+ 1643us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:28.657244 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:28.660426 (+ 3182us) server_negotiation.cc:407] Connection header received
0806 01:13:28.674821 (+ 14395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:28.674851 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:28.675616 (+ 765us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:28.676247 (+ 631us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:28.680375 (+ 4128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:28.682922 (+ 2547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:28.685322 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:28.686823 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:28.687109 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:28.695350 (+ 8241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:28.695417 (+ 67us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:28.695460 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:28.695549 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:28.702877 (+ 7328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:28.703973 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:28.703983 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:28.703993 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:28.704271 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:28.704741 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:28.704750 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:28.704756 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:28.705794 (+ 1038us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:28.706366 (+ 572us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:28.707275 (+ 909us) server_negotiation.cc:299] Negotiation successful
0806 01:13:28.707804 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1189,"thread_start_us":735,"threads_started":1}
I20260806 01:13:28.711344 26683 consensus_queue.cc:237] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec2c52fde404f0a97ec90d96ce67ceb" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39087 } }
I20260806 01:13:28.713452 26675 sys_catalog.cc:564] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:28.719204 26686 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bec2c52fde404f0a97ec90d96ce67ceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec2c52fde404f0a97ec90d96ce67ceb" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39087 } } }
I20260806 01:13:28.719272 26685 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [sys.catalog]: SysCatalogTable state changed. Reason: New leader bec2c52fde404f0a97ec90d96ce67ceb. Latest consensus state: current_term: 1 leader_uuid: "bec2c52fde404f0a97ec90d96ce67ceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec2c52fde404f0a97ec90d96ce67ceb" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39087 } } }
I20260806 01:13:28.720712 26686 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:28.720934 26685 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:28.724869 26693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:13:28.734170 26693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:13:28.749073 26693 catalog_manager.cc:1347] Generated new cluster ID: 337449f384cd45c8a93881d219e79cd7
I20260806 01:13:28.749248 26693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:13:28.784754 26693 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:13:28.786437 26693 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:13:28.800158 26693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bec2c52fde404f0a97ec90d96ce67ceb: Generated new TSK 0
I20260806 01:13:28.801347 26693 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:13:28.863313 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39087
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:29.389261 26714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:29.389750 26714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:29.389937 26714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:29.400157 26714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:29.400485 26714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:29.400763 26714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:13:29.413853 26714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39087
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:29.415652 26714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:29.417325 26714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:29.434646 26724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:29.435554 26731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:29.435607 26734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:29.436523 26714 server_base.cc:1034] running on GCE node
I20260806 01:13:29.437390 26714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:29.438675 26714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:29.439920 26714 hybrid_clock.cc:648] HybridClock initialized: now 1785978809439877 us; error 40 us; skew 500 ppm
I20260806 01:13:29.440565 26714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:29 dist-test-slave-7nm7 krb5kdc[26557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978809, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:29.444887 26714 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:13:29.446808 26714 webserver.cc:460] Webserver started at http://127.25.236.129:46295/ using document root <none> and password file <none>
I20260806 01:13:29.447546 26714 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:29.447669 26714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:29.447974 26714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:29.451617 26714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "c82166f13ef740faa008a951818e6532"
format_stamp: "Formatted at 2026-08-06 01:13:29 on dist-test-slave-7nm7"
server_key: "854e34c0e5cb4e353cdec187c52446fe"
server_key_iv: "a2fc66514ce27452fe6ddf4682bf65fb"
server_key_version: "encryptionkey@0"
I20260806 01:13:29.452397 26714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "c82166f13ef740faa008a951818e6532"
format_stamp: "Formatted at 2026-08-06 01:13:29 on dist-test-slave-7nm7"
server_key: "854e34c0e5cb4e353cdec187c52446fe"
server_key_iv: "a2fc66514ce27452fe6ddf4682bf65fb"
server_key_version: "encryptionkey@0"
I20260806 01:13:29.458942 26714 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 01:13:29.463616 26743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:29.465193 26714 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:13:29.465426 26714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "c82166f13ef740faa008a951818e6532"
format_stamp: "Formatted at 2026-08-06 01:13:29 on dist-test-slave-7nm7"
server_key: "854e34c0e5cb4e353cdec187c52446fe"
server_key_iv: "a2fc66514ce27452fe6ddf4682bf65fb"
server_key_version: "encryptionkey@0"
I20260806 01:13:29.465739 26714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:29.496389 26714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:29.501860 26714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:29.502218 26714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:29.504072 26714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:13:29.506215 26714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:29.506422 26714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 01:13:29.506680 26714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:29.506788 26714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:29.574512 26714 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:45481
I20260806 01:13:29.574584 26904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:45481 every 8 connection(s)
I20260806 01:13:29.576377 26714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:13:29.583244 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 26714
I20260806 01:13:29.583663 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:13:29.584379 26546 external_mini_cluster.cc:1428] Setting key af641eeacfe1641f16f4ebadef0e6cd4
Aug 06 01:13:29 dist-test-slave-7nm7 krb5kdc[26557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978809, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:29.606108 26909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:29.581360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:48395 (local address 127.25.236.190:39087)
0806 01:13:29.581846 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:29.581851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:29.583337 (+ 1486us) server_negotiation.cc:407] Connection header received
0806 01:13:29.586044 (+ 2707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:29.586050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:29.586135 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:29.586300 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:29.588188 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:29.588952 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:29.591315 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:29.591819 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:29.591960 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:29.597914 (+ 5954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:29.597925 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:29.597933 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:29.597987 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:29.602192 (+ 4205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:29.603035 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:29.603044 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:29.603049 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:29.603302 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:29.603774 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:29.603782 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:29.603788 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:29.604217 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:29.604397 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:29.605272 (+ 875us) server_negotiation.cc:299] Negotiation successful
0806 01:13:29.605531 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":245,"threads_started":1}
I20260806 01:13:29.606899 26910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:29.581978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39087 (local address 127.25.236.129:48395)
0806 01:13:29.583135 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:29.583182 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:29.585778 (+ 2596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:29.586593 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:29.586602 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:29.587409 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:29.588007 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:29.588025 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:29.589168 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:29.589182 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:29.590960 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:29.590973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:29.592156 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:29.592203 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:29.592702 (+ 499us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:29.593906 (+ 1204us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:29.593941 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:29.597574 (+ 3633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:29.602394 (+ 4820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:29.602402 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:29.602415 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:29.602895 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:29.603428 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:29.603431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:29.603433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:29.603636 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:29.604396 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:29.604403 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:29.604770 (+ 367us) client_negotiation.cc:769] Sending connection context
0806 01:13:29.605076 (+ 306us) client_negotiation.cc:240] Negotiation successful
0806 01:13:29.605580 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"thread_start_us":490,"threads_started":1}
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
I20260806 01:13:29.608922 26905 heartbeater.cc:347] Connected to a master server at 127.25.236.190:39087
I20260806 01:13:29.609408 26905 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
I20260806 01:13:29.610536 26905 heartbeater.cc:511] Master 127.25.236.190:39087 requested a full tablet report, sending...
I20260806 01:13:29.614600 26628 ts_manager.cc:194] Registered new tserver with Master: c82166f13ef740faa008a951818e6532 (127.25.236.129:45481)
I20260806 01:13:29.617120 26628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:48395
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:13:29.638438 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39087
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:30.190543 26915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:30.190824 26915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:30.191026 26915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:30.199982 26915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:30.200260 26915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:30.200474 26915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:13:30.214602 26915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39087
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:30.216126 26915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:30.217682 26915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:30.236418 26920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:30.238122 26923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:30.238535 26921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:30.239501 26915 server_base.cc:1034] running on GCE node
I20260806 01:13:30.240211 26915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:30.241340 26915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:30.242748 26915 hybrid_clock.cc:648] HybridClock initialized: now 1785978810242612 us; error 58 us; skew 500 ppm
I20260806 01:13:30.243362 26915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:30 dist-test-slave-7nm7 krb5kdc[26557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978810, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:30.247605 26915 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:13:30.249563 26915 webserver.cc:460] Webserver started at http://127.25.236.130:39199/ using document root <none> and password file <none>
I20260806 01:13:30.250661 26915 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:30.250833 26915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:30.251169 26915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:30.255108 26915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "81897bca93a3411786919a48d230c4a7"
format_stamp: "Formatted at 2026-08-06 01:13:30 on dist-test-slave-7nm7"
server_key: "bd2e1eae1994817c8af3f7df501103a7"
server_key_iv: "28cfd6142a9aeed2b19498c56cf7f37b"
server_key_version: "encryptionkey@0"
I20260806 01:13:30.256083 26915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "81897bca93a3411786919a48d230c4a7"
format_stamp: "Formatted at 2026-08-06 01:13:30 on dist-test-slave-7nm7"
server_key: "bd2e1eae1994817c8af3f7df501103a7"
server_key_iv: "28cfd6142a9aeed2b19498c56cf7f37b"
server_key_version: "encryptionkey@0"
I20260806 01:13:30.263054 26915 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 01:13:30.267251 26934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:30.268996 26915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260806 01:13:30.269280 26915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "81897bca93a3411786919a48d230c4a7"
format_stamp: "Formatted at 2026-08-06 01:13:30 on dist-test-slave-7nm7"
server_key: "bd2e1eae1994817c8af3f7df501103a7"
server_key_iv: "28cfd6142a9aeed2b19498c56cf7f37b"
server_key_version: "encryptionkey@0"
I20260806 01:13:30.269611 26915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:30.290524 26915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:30.296543 26915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:30.297016 26915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:30.298744 26915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:13:30.300900 26915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:30.301084 26915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:30.301255 26915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:30.301338 26915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:30.364162 26915 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:42695
I20260806 01:13:30.364406 27108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:42695 every 8 connection(s)
I20260806 01:13:30.365677 26915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:13:30.367226 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 26915
I20260806 01:13:30.367443 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:13:30.367985 26546 external_mini_cluster.cc:1428] Setting key 9704348433beab56a0d9ddf57a3b298d
Aug 06 01:13:30 dist-test-slave-7nm7 krb5kdc[26557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978810, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
I20260806 01:13:30.400772 27111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:30.369013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:38087 (local address 127.25.236.190:39087)
0806 01:13:30.374923 (+ 5910us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:30.374927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:30.374946 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:13:30.375535 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:30.375539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:30.375606 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:30.375711 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:30.378385 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:30.379558 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:30.382346 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:30.383051 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:30.383180 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:30.389345 (+ 6165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:30.389375 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:30.389384 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:30.389448 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:30.395071 (+ 5623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:30.397218 (+ 2147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:30.397224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:30.397226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:30.397394 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:30.398106 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:30.398118 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:30.398121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:30.398607 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:30.398739 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:30.400124 (+ 1385us) server_negotiation.cc:299] Negotiation successful
0806 01:13:30.400337 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5799,"thread_start_us":5678,"threads_started":1}
I20260806 01:13:30.403573 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:30.369756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39087 (local address 127.25.236.130:38087)
0806 01:13:30.371520 (+ 1764us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:30.371658 (+ 138us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:30.375169 (+ 3511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:30.376039 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:30.376054 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:30.377023 (+ 969us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:30.378156 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:30.378187 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:30.379800 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:30.379804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:30.382062 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:30.382082 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:30.383365 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:30.383376 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:30.383832 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:30.384839 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:30.384877 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:30.388816 (+ 3939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:30.395548 (+ 6732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:30.395561 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:30.395585 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:30.396912 (+ 1327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:30.397558 (+ 646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:30.397562 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:30.397565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:30.397951 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:30.398828 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:30.398839 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:30.399642 (+ 803us) client_negotiation.cc:769] Sending connection context
0806 01:13:30.400087 (+ 445us) client_negotiation.cc:240] Negotiation successful
0806 01:13:30.401396 (+ 1309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1400,"thread_start_us":964,"threads_started":1}
I20260806 01:13:30.405613 27109 heartbeater.cc:347] Connected to a master server at 127.25.236.190:39087
I20260806 01:13:30.406030 27109 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:30.407543 27109 heartbeater.cc:511] Master 127.25.236.190:39087 requested a full tablet report, sending...
I20260806 01:13:30.409353 26628 ts_manager.cc:194] Registered new tserver with Master: 81897bca93a3411786919a48d230c4a7 (127.25.236.130:42695)
I20260806 01:13:30.410223 26628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:38087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:13:30.430091 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39087
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:13:30.622166 26905 heartbeater.cc:503] Master 127.25.236.190:39087 was elected leader, sending a full tablet report...
W20260806 01:13:31.007552 27116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:31.007979 27116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:31.008219 27116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:31.018517 27116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:31.018697 27116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:31.018954 27116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:13:31.031251 27116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39087
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:31.032851 27116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:31.034541 27116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:31.051723 27121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:31.051957 27122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:31.052397 27124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:31.054529 27116 server_base.cc:1034] running on GCE node
I20260806 01:13:31.055207 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:31.056463 27116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:31.057866 27116 hybrid_clock.cc:648] HybridClock initialized: now 1785978811057718 us; error 124 us; skew 500 ppm
I20260806 01:13:31.058452 27116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:31 dist-test-slave-7nm7 krb5kdc[26557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978811, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:31.063954 27116 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:13:31.065920 27116 webserver.cc:460] Webserver started at http://127.25.236.131:44497/ using document root <none> and password file <none>
I20260806 01:13:31.066866 27116 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:31.067065 27116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:31.067490 27116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:31.071852 27116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "4d33124ceb474da282128ddd3dff9c2d"
format_stamp: "Formatted at 2026-08-06 01:13:31 on dist-test-slave-7nm7"
server_key: "d588c24db69009c335d390840b55508e"
server_key_iv: "49dd9e3c060d75ad9995ef2e4e54ed43"
server_key_version: "encryptionkey@0"
I20260806 01:13:31.072793 27116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "4d33124ceb474da282128ddd3dff9c2d"
format_stamp: "Formatted at 2026-08-06 01:13:31 on dist-test-slave-7nm7"
server_key: "d588c24db69009c335d390840b55508e"
server_key_iv: "49dd9e3c060d75ad9995ef2e4e54ed43"
server_key_version: "encryptionkey@0"
I20260806 01:13:31.080905 27116 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.008s
I20260806 01:13:31.085500 27138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:31.087486 27116 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 01:13:31.087740 27116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "4d33124ceb474da282128ddd3dff9c2d"
format_stamp: "Formatted at 2026-08-06 01:13:31 on dist-test-slave-7nm7"
server_key: "d588c24db69009c335d390840b55508e"
server_key_iv: "49dd9e3c060d75ad9995ef2e4e54ed43"
server_key_version: "encryptionkey@0"
I20260806 01:13:31.087981 27116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:31.115549 27116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:31.122928 27116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:31.123369 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:31.124934 27116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:13:31.127240 27116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:31.127434 27116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:31.127596 27116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:31.127725 27116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:31.184408 27116 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:41011
I20260806 01:13:31.184537 27299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:41011 every 8 connection(s)
I20260806 01:13:31.186141 27116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:13:31.196249 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 27116
I20260806 01:13:31.196887 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:13:31.197628 26546 external_mini_cluster.cc:1428] Setting key ffa2e8679cba23e91ff9baae217f7aa4
Aug 06 01:13:31 dist-test-slave-7nm7 krb5kdc[26557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978811, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:31.216291 27304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:31.189929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:38909 (local address 127.25.236.190:39087)
0806 01:13:31.190778 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:31.190783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:31.192510 (+ 1727us) server_negotiation.cc:407] Connection header received
0806 01:13:31.195874 (+ 3364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:31.195882 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:31.196010 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:31.196179 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:31.199465 (+ 3286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.200656 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.203027 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.203789 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.203900 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:31.208764 (+ 4864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:31.208770 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:31.208773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:31.208809 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:31.212206 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:31.213054 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:31.213058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:31.213061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:31.213198 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:31.213655 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:31.213660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:31.213662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:31.214204 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:31.214391 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:31.215495 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0806 01:13:31.215713 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":483,"threads_started":1}
I20260806 01:13:31.217593 27305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:31.190744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39087 (local address 127.25.236.131:38909)
0806 01:13:31.192325 (+ 1581us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:31.192368 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:31.195182 (+ 2814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:31.196802 (+ 1620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:31.196821 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:31.197928 (+ 1107us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:31.199107 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:31.199137 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.200860 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.200865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:31.202826 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:31.202849 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.204024 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.204029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:31.204479 (+ 450us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:31.205552 (+ 1073us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:31.205581 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:31.208508 (+ 2927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:31.212359 (+ 3851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:31.212367 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:31.212381 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:31.212927 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:31.213301 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:31.213305 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:31.213307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:31.213506 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:31.214408 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:31.214416 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:31.214950 (+ 534us) client_negotiation.cc:769] Sending connection context
0806 01:13:31.215303 (+ 353us) client_negotiation.cc:240] Negotiation successful
0806 01:13:31.215882 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1165,"thread_start_us":692,"threads_started":1}
I20260806 01:13:31.219451 27301 heartbeater.cc:347] Connected to a master server at 127.25.236.190:39087
I20260806 01:13:31.219837 27301 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:31.220821 27301 heartbeater.cc:511] Master 127.25.236.190:39087 requested a full tablet report, sending...
I20260806 01:13:31.222694 26628 ts_manager.cc:194] Registered new tserver with Master: 4d33124ceb474da282128ddd3dff9c2d (127.25.236.131:41011)
I20260806 01:13:31.224217 26628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:38909
I20260806 01:13:31.229485 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:13:31 dist-test-slave-7nm7 krb5kdc[26557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978811, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:13:31 dist-test-slave-7nm7 krb5kdc[26557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978811, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:13:31.291616 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:31.270594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.25.236.129:45481)
0806 01:13:31.271582 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:31.271590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:31.271666 (+ 76us) server_negotiation.cc:407] Connection header received
0806 01:13:31.271954 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:31.271963 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:31.272204 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:31.272384 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:31.274724 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.275867 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.278144 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.279107 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.279278 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:31.284212 (+ 4934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:31.284226 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:31.284242 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:31.284278 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:31.288073 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:31.288985 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:31.288992 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:31.288995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:31.289151 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:31.289855 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:31.289859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:31.289861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:31.290268 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:31.290675 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:31.290887 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 01:13:31.291100 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":488,"threads_started":1}
W20260806 01:13:31.293555 26785 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50758
Aug 06 01:13:31 dist-test-slave-7nm7 krb5kdc[26557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:13:31 dist-test-slave-7nm7 krb5kdc[26557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:13:31.360443 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:31.340052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50772 (local address 127.25.236.129:45481)
0806 01:13:31.340440 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:31.340450 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:31.340877 (+ 427us) server_negotiation.cc:407] Connection header received
0806 01:13:31.341114 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:31.341119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:31.341224 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:31.341336 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:31.343497 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.344386 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.345804 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.346825 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.347021 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:31.353997 (+ 6976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:31.354003 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:31.354007 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:31.354055 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:31.357033 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:31.358506 (+ 1473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:31.358511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:31.358513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:31.358662 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:31.359062 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:31.359066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:31.359068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:31.359376 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:31.359567 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:31.360003 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 01:13:31.360184 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260806 01:13:31.362789 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 26714
I20260806 01:13:31.385222 27313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:31.364024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50788 (local address 127.25.236.129:45481)
0806 01:13:31.364296 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:31.364303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:31.364606 (+ 303us) server_negotiation.cc:407] Connection header received
0806 01:13:31.364763 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:31.364767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:31.364826 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:31.364906 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:31.367613 (+ 2707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.368700 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.370659 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.371527 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.371675 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:31.375798 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:31.375804 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:31.375808 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:31.375847 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:31.381563 (+ 5716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:31.382602 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:31.382607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:31.382609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:31.382863 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:31.383409 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:31.383413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:31.383415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:31.383804 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:31.384032 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:31.384420 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 01:13:31.384621 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:13:31.386193 26900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:31.413395 27109 heartbeater.cc:503] Master 127.25.236.190:39087 was elected leader, sending a full tablet report...
I20260806 01:13:31.489538 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 26714
I20260806 01:13:31.520272 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 26915
Aug 06 01:13:31 dist-test-slave-7nm7 krb5kdc[26557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:13:31.539909 27322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:31.521495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46448 (local address 127.25.236.130:42695)
0806 01:13:31.522325 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:31.522333 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:31.522363 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:13:31.522433 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:31.522441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:31.522668 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:31.522868 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:31.524226 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.525447 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.527223 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.527841 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.527963 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:31.533597 (+ 5634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:31.533620 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:31.533636 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:31.533682 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:31.537060 (+ 3378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:31.537838 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:31.537846 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:31.537848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:31.537997 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:31.538511 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:31.538515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:31.538517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:31.538889 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:31.539137 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:31.539399 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 01:13:31.539610 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":351,"threads_started":1}
I20260806 01:13:31.541332 27104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:31.630949 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 26915
I20260806 01:13:31.657426 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27116
Aug 06 01:13:31 dist-test-slave-7nm7 krb5kdc[26557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:13:31.682016 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:31.659059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.25.236.131:41011)
0806 01:13:31.659634 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:31.659642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:31.659673 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:13:31.659736 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:31.659742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:31.659953 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:31.660083 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:31.661599 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.662754 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.664055 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:31.664624 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:31.664752 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:31.670441 (+ 5689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:31.670472 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:31.670503 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:31.670577 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:31.676028 (+ 5451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:31.677406 (+ 1378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:31.677418 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:31.677423 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:31.677643 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:31.679993 (+ 2350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:31.680000 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:31.680003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:31.680449 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:31.680762 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:31.681227 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 01:13:31.681564 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":281,"threads_started":1}
I20260806 01:13:31.683789 27294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:31.793061 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 27116
I20260806 01:13:31.820781 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 26576
I20260806 01:13:31.822082 26670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:31.932071 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 26576
2026-08-06T01:13:31Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (6516 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:13:32 dist-test-slave-7nm7 krb5kdc[27330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:13:32 dist-test-slave-7nm7 krb5kdc[27330](info): set up 2 sockets
Aug 06 01:13:32 dist-test-slave-7nm7 krb5kdc[27330](info): commencing operation
krb5kdc: starting...
W20260806 01:13:33.299922 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:13:33 dist-test-slave-7nm7 krb5kdc[27330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:13:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:13:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:13:33.536280 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:33751
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:33751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:33.909802 27347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:33.910216 27347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:33.910404 27347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:33.919675 27347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:33.919852 27347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:33.919962 27347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:33.920063 27347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:33.920140 27347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:33.931289 27347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:33751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:33751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:33.933040 27347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:33.934763 27347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:33.947894 27354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:33.947969 27355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:33.948004 27360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:33.948675 27347 server_base.cc:1034] running on GCE node
I20260806 01:13:33.949605 27347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:33.951236 27347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:33.952613 27347 hybrid_clock.cc:648] HybridClock initialized: now 1785978813952543 us; error 53 us; skew 500 ppm
I20260806 01:13:33.952996 27347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:33 dist-test-slave-7nm7 krb5kdc[27330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978813, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:33.965059 27347 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:13:33.966791 27347 webserver.cc:460] Webserver started at http://127.25.236.190:46779/ using document root <none> and password file <none>
I20260806 01:13:33.967526 27347 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:33.967675 27347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:33.968011 27347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:33.971438 27347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "0ff7253efffd4a10ac56c9bb4cd709c1"
format_stamp: "Formatted at 2026-08-06 01:13:33 on dist-test-slave-7nm7"
server_key: "39e886e198f39583f799af432b6228a3"
server_key_iv: "81c322951874149061717399f5a26ec8"
server_key_version: "encryptionkey@0"
I20260806 01:13:33.972362 27347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "0ff7253efffd4a10ac56c9bb4cd709c1"
format_stamp: "Formatted at 2026-08-06 01:13:33 on dist-test-slave-7nm7"
server_key: "39e886e198f39583f799af432b6228a3"
server_key_iv: "81c322951874149061717399f5a26ec8"
server_key_version: "encryptionkey@0"
I20260806 01:13:33.979183 27347 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 01:13:33.983371 27369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:33.984959 27347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.004s
I20260806 01:13:33.985176 27347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "0ff7253efffd4a10ac56c9bb4cd709c1"
format_stamp: "Formatted at 2026-08-06 01:13:33 on dist-test-slave-7nm7"
server_key: "39e886e198f39583f799af432b6228a3"
server_key_iv: "81c322951874149061717399f5a26ec8"
server_key_version: "encryptionkey@0"
I20260806 01:13:33.985414 27347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:33.998466 27347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:34.003880 27347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:34.004344 27347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:34.028689 27347 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:33751
I20260806 01:13:34.028710 27447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:33751 every 8 connection(s)
I20260806 01:13:34.030157 27347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:13:34.034070 27448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:34.040381 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 27347
I20260806 01:13:34.040659 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:13:34.041337 26546 external_mini_cluster.cc:1428] Setting key 13c2accbb2d9bfa9ddb3856901480289
I20260806 01:13:34.048849 27448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1: Bootstrap starting.
I20260806 01:13:34.053074 27448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:34.054478 27448 log.cc:824] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:34.057547 27448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1: No bootstrap required, opened a new log
Aug 06 01:13:34 dist-test-slave-7nm7 krb5kdc[27330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:34.068519 27448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff7253efffd4a10ac56c9bb4cd709c1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33751 } }
I20260806 01:13:34.069026 27448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:34.069161 27448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ff7253efffd4a10ac56c9bb4cd709c1, State: Initialized, Role: FOLLOWER
I20260806 01:13:34.069777 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:34.043558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.25.236.190:33751)
0806 01:13:34.044470 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:34.044482 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:34.044529 (+ 47us) server_negotiation.cc:407] Connection header received
0806 01:13:34.046658 (+ 2129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:34.046683 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:34.047102 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:34.047465 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:34.050186 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:34.052052 (+ 1866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:34.054126 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:34.055039 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:34.055292 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:34.059260 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:34.059271 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:34.059293 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:34.059356 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:34.064366 (+ 5010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:34.065454 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:34.065468 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:34.065478 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:34.065734 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:34.066135 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:34.066142 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:34.066145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:34.067047 (+ 902us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:34.067360 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:34.067841 (+ 481us) server_negotiation.cc:299] Negotiation successful
0806 01:13:34.068316 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":399,"threads_started":1}
I20260806 01:13:34.069877 27448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff7253efffd4a10ac56c9bb4cd709c1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33751 } }
I20260806 01:13:34.070183 27448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:13:34.070381 27448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:13:34.070591 27448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:34.073406 27448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff7253efffd4a10ac56c9bb4cd709c1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33751 } }
I20260806 01:13:34.073868 27448 leader_election.cc:302] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ff7253efffd4a10ac56c9bb4cd709c1; no voters:
I20260806 01:13:34.074556 27448 leader_election.cc:288] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:13:34.074682 27454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:34.075557 27454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [term 1 LEADER]: Becoming Leader. State: Replica: 0ff7253efffd4a10ac56c9bb4cd709c1, State: Running, Role: LEADER
I20260806 01:13:34.076107 27454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff7253efffd4a10ac56c9bb4cd709c1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33751 } }
I20260806 01:13:34.076804 27448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:34.079228 27457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ff7253efffd4a10ac56c9bb4cd709c1. Latest consensus state: current_term: 1 leader_uuid: "0ff7253efffd4a10ac56c9bb4cd709c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff7253efffd4a10ac56c9bb4cd709c1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33751 } } }
I20260806 01:13:34.079196 27455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ff7253efffd4a10ac56c9bb4cd709c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff7253efffd4a10ac56c9bb4cd709c1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33751 } } }
I20260806 01:13:34.079756 27457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:34.079756 27455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:34.081763 27460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:13:34.087633 27460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:13:34.096318 27460 catalog_manager.cc:1347] Generated new cluster ID: 609b5b0c3b69433eafb8fffda3042001
I20260806 01:13:34.096446 27460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:13:34.138223 27460 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:13:34.139724 27460 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:13:34.160322 27460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ff7253efffd4a10ac56c9bb4cd709c1: Generated new TSK 0
I20260806 01:13:34.161414 27460 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:13:34.226462 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33751
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:34.597846 27482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:34.598284 27482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:34.598457 27482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:34.606344 27482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:34.606503 27482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:34.606694 27482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:13:34.616990 27482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33751
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:34.618512 27482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:34.619982 27482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:34.634853 27495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:34.634941 27489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:34.634897 27490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:34.637326 27482 server_base.cc:1034] running on GCE node
I20260806 01:13:34.638042 27482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:34.639160 27482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:34.640419 27482 hybrid_clock.cc:648] HybridClock initialized: now 1785978814640343 us; error 61 us; skew 500 ppm
I20260806 01:13:34.640797 27482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:34 dist-test-slave-7nm7 krb5kdc[27330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978814, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:34.644198 27482 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:13:34.645870 27482 webserver.cc:460] Webserver started at http://127.25.236.129:40317/ using document root <none> and password file <none>
I20260806 01:13:34.646695 27482 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:34.646852 27482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:34.647181 27482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:34.650414 27482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "a62dc3a7121b43fb9af81c15cf308d8b"
format_stamp: "Formatted at 2026-08-06 01:13:34 on dist-test-slave-7nm7"
server_key: "08d1022050e1e64af1c5cb86799a76df"
server_key_iv: "4494446e226243ddb83a3e79fd505add"
server_key_version: "encryptionkey@0"
I20260806 01:13:34.651285 27482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "a62dc3a7121b43fb9af81c15cf308d8b"
format_stamp: "Formatted at 2026-08-06 01:13:34 on dist-test-slave-7nm7"
server_key: "08d1022050e1e64af1c5cb86799a76df"
server_key_iv: "4494446e226243ddb83a3e79fd505add"
server_key_version: "encryptionkey@0"
I20260806 01:13:34.657063 27482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 01:13:34.660993 27506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:34.662655 27482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:13:34.662849 27482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "a62dc3a7121b43fb9af81c15cf308d8b"
format_stamp: "Formatted at 2026-08-06 01:13:34 on dist-test-slave-7nm7"
server_key: "08d1022050e1e64af1c5cb86799a76df"
server_key_iv: "4494446e226243ddb83a3e79fd505add"
server_key_version: "encryptionkey@0"
I20260806 01:13:34.663081 27482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:34.684713 27482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:34.689476 27482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:34.689790 27482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:34.690879 27482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:13:34.692299 27482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:34.692402 27482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:34.692564 27482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:34.692636 27482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:34.728072 27482 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:33223
I20260806 01:13:34.728163 27672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:33223 every 8 connection(s)
I20260806 01:13:34.729429 27482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:13:34.731583 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 27482
I20260806 01:13:34.731796 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:13:34.732532 26546 external_mini_cluster.cc:1428] Setting key 22fb280a7acbcc60dbefe1ac53b05cf5
Aug 06 01:13:34 dist-test-slave-7nm7 krb5kdc[27330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978814, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:34.750941 27677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:34.732243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:60981 (local address 127.25.236.190:33751)
0806 01:13:34.732797 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:34.732803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:34.733840 (+ 1037us) server_negotiation.cc:407] Connection header received
0806 01:13:34.735866 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:34.735871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:34.735962 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:34.736089 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:34.737726 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:34.738564 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:34.740119 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:34.740546 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:34.740687 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:34.745098 (+ 4411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:34.745104 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:34.745107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:34.745143 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:34.747718 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:34.748489 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:34.748495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:34.748498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:34.748622 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:34.749068 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:34.749074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:34.749077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:34.749356 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:34.749520 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:34.750302 (+ 782us) server_negotiation.cc:299] Negotiation successful
0806 01:13:34.750542 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":239,"threads_started":1}
I20260806 01:13:34.751977 27680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:34.732824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:33751 (local address 127.25.236.129:60981)
0806 01:13:34.733657 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:34.733705 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:34.735567 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:34.736371 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:34.736380 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:34.736937 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:34.737540 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:34.737559 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:34.738714 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:34.738731 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:34.739968 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:34.739980 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:34.740809 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:34.740817 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:34.741050 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:34.741807 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:34.741836 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:34.744836 (+ 3000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:34.747871 (+ 3035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:34.747881 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:34.747900 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:34.748352 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:34.748746 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:34.748751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:34.748755 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:34.748947 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:34.749492 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:34.749501 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:34.749880 (+ 379us) client_negotiation.cc:769] Sending connection context
0806 01:13:34.750168 (+ 288us) client_negotiation.cc:240] Negotiation successful
0806 01:13:34.750784 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":348,"threads_started":1}
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
I20260806 01:13:34.753737 27673 heartbeater.cc:347] Connected to a master server at 127.25.236.190:33751
I20260806 01:13:34.754127 27673 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:34.754988 27673 heartbeater.cc:511] Master 127.25.236.190:33751 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
I20260806 01:13:34.757633 27389 ts_manager.cc:194] Registered new tserver with Master: a62dc3a7121b43fb9af81c15cf308d8b (127.25.236.129:33223)
I20260806 01:13:34.759371 27389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:60981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:13:34.787847 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33751
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:35.135993 27683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:35.136384 27683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:35.136560 27683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:35.145222 27683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:35.145421 27683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:35.145618 27683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:13:35.156224 27683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33751
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:35.157653 27683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:35.159181 27683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:35.173331 27693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:35.173406 27698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:35.173856 27692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:35.174842 27683 server_base.cc:1034] running on GCE node
I20260806 01:13:35.175524 27683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:35.176645 27683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:35.177918 27683 hybrid_clock.cc:648] HybridClock initialized: now 1785978815177833 us; error 81 us; skew 500 ppm
I20260806 01:13:35.178357 27683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:35 dist-test-slave-7nm7 krb5kdc[27330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978815, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:35.182544 27683 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:13:35.184330 27683 webserver.cc:460] Webserver started at http://127.25.236.130:36473/ using document root <none> and password file <none>
I20260806 01:13:35.185096 27683 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:35.185251 27683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:35.185582 27683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:35.188956 27683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "545d82876e78442080403f933b3566bf"
format_stamp: "Formatted at 2026-08-06 01:13:35 on dist-test-slave-7nm7"
server_key: "7a84306677537151a265c57ce9ca9e4c"
server_key_iv: "26b9c410fb92335aa26ea91274c0b6a0"
server_key_version: "encryptionkey@0"
I20260806 01:13:35.189859 27683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "545d82876e78442080403f933b3566bf"
format_stamp: "Formatted at 2026-08-06 01:13:35 on dist-test-slave-7nm7"
server_key: "7a84306677537151a265c57ce9ca9e4c"
server_key_iv: "26b9c410fb92335aa26ea91274c0b6a0"
server_key_version: "encryptionkey@0"
I20260806 01:13:35.195413 27683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:13:35.199373 27709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:35.201263 27683 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:13:35.201483 27683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "545d82876e78442080403f933b3566bf"
format_stamp: "Formatted at 2026-08-06 01:13:35 on dist-test-slave-7nm7"
server_key: "7a84306677537151a265c57ce9ca9e4c"
server_key_iv: "26b9c410fb92335aa26ea91274c0b6a0"
server_key_version: "encryptionkey@0"
I20260806 01:13:35.201725 27683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:35.233289 27683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:35.239148 27683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:35.239488 27683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:35.240741 27683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:13:35.242277 27683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:35.242389 27683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 01:13:35.242545 27683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:35.242630 27683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:35.280228 27683 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:41081
I20260806 01:13:35.280385 27874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:41081 every 8 connection(s)
I20260806 01:13:35.281661 27683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:13:35.291937 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 27683
I20260806 01:13:35.292299 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:13:35.293303 26546 external_mini_cluster.cc:1428] Setting key 50ae1a4c5d795b7b884fef56c3e0b466
Aug 06 01:13:35 dist-test-slave-7nm7 krb5kdc[27330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978815, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:35.304958 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:35.284708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:49137 (local address 127.25.236.190:33751)
0806 01:13:35.285208 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:35.285216 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:35.286074 (+ 858us) server_negotiation.cc:407] Connection header received
0806 01:13:35.288257 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:35.288262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:35.288334 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:35.288436 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:35.290139 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:35.290942 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:35.292767 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:35.293648 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:35.293800 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:35.298433 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:35.298438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:35.298442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:35.298476 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:35.301353 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:35.302342 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:35.302346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:35.302348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:35.302476 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:35.302980 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:35.302983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:35.302985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:35.303254 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:35.303430 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:35.304156 (+ 726us) server_negotiation.cc:299] Negotiation successful
0806 01:13:35.304439 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":225,"threads_started":1}
I20260806 01:13:35.305792 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:35.285093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:33751 (local address 127.25.236.130:49137)
0806 01:13:35.285903 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:35.285943 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:35.287971 (+ 2028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:35.288661 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:35.288671 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:35.289359 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:35.289933 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:35.289950 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:35.291120 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:35.291124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:35.292488 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:35.292510 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:35.293892 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:35.293898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:35.294221 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:35.295010 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:35.295040 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:35.298074 (+ 3034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:35.301510 (+ 3436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:35.301522 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:35.301543 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:35.302204 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:35.302585 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:35.302593 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:35.302599 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:35.302867 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:35.303408 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:35.303414 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:35.303756 (+ 342us) client_negotiation.cc:769] Sending connection context
0806 01:13:35.303985 (+ 229us) client_negotiation.cc:240] Negotiation successful
0806 01:13:35.304416 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":318,"threads_started":1}
I20260806 01:13:35.307511 27876 heartbeater.cc:347] Connected to a master server at 127.25.236.190:33751
I20260806 01:13:35.307837 27876 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:35.308590 27876 heartbeater.cc:511] Master 127.25.236.190:33751 requested a full tablet report, sending...
I20260806 01:13:35.310645 27389 ts_manager.cc:194] Registered new tserver with Master: 545d82876e78442080403f933b3566bf (127.25.236.130:41081)
I20260806 01:13:35.311940 27389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:49137
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:13:35.346560 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33751
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:37527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:13:35.762997 27673 heartbeater.cc:503] Master 127.25.236.190:33751 was elected leader, sending a full tablet report...
W20260806 01:13:35.809665 27884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:35.809963 27884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:35.810065 27884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:35.817756 27884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:35.817884 27884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:35.818003 27884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:13:35.828536 27884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:37527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33751
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:35.829813 27884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:35.831480 27884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:35.845506 27892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:35.845563 27893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:35.845479 27898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:35.846643 27884 server_base.cc:1034] running on GCE node
I20260806 01:13:35.847254 27884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:35.848271 27884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:35.849494 27884 hybrid_clock.cc:648] HybridClock initialized: now 1785978815849421 us; error 56 us; skew 500 ppm
I20260806 01:13:35.849879 27884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:35 dist-test-slave-7nm7 krb5kdc[27330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978815, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:35.853721 27884 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:13:35.855923 27884 webserver.cc:460] Webserver started at http://127.25.236.131:44123/ using document root <none> and password file <none>
I20260806 01:13:35.856662 27884 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:35.856798 27884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:35.857154 27884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:35.860822 27884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "26a9ac3402a742db8b1b0d64ece7b4ca"
format_stamp: "Formatted at 2026-08-06 01:13:35 on dist-test-slave-7nm7"
server_key: "01ba95d35bef36f0d96abb53195ef8f2"
server_key_iv: "d6716e18f0174d17ac9b63cfb163984d"
server_key_version: "encryptionkey@0"
I20260806 01:13:35.861817 27884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "26a9ac3402a742db8b1b0d64ece7b4ca"
format_stamp: "Formatted at 2026-08-06 01:13:35 on dist-test-slave-7nm7"
server_key: "01ba95d35bef36f0d96abb53195ef8f2"
server_key_iv: "d6716e18f0174d17ac9b63cfb163984d"
server_key_version: "encryptionkey@0"
I20260806 01:13:35.867869 27884 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 01:13:35.871910 27908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:35.873528 27884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:13:35.873744 27884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "26a9ac3402a742db8b1b0d64ece7b4ca"
format_stamp: "Formatted at 2026-08-06 01:13:35 on dist-test-slave-7nm7"
server_key: "01ba95d35bef36f0d96abb53195ef8f2"
server_key_iv: "d6716e18f0174d17ac9b63cfb163984d"
server_key_version: "encryptionkey@0"
I20260806 01:13:35.873979 27884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:35.893034 27884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:35.897459 27884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:35.897823 27884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:35.898990 27884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:13:35.900985 27884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:35.901263 27884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:35.901520 27884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:35.901603 27884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:35.940793 27884 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:46851
I20260806 01:13:35.940853 28076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:46851 every 8 connection(s)
I20260806 01:13:35.942119 27884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:13:35.943562 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 27884
I20260806 01:13:35.943786 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:13:35.944268 26546 external_mini_cluster.cc:1428] Setting key 2b90bff971c51cdaf34091793374d2d8
Aug 06 01:13:35 dist-test-slave-7nm7 krb5kdc[27330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978815, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:35.968268 28082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:35.945794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:33751 (local address 127.25.236.131:38459)
0806 01:13:35.946867 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:35.946939 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:35.949887 (+ 2948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:35.950754 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:35.950763 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:35.951466 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:35.952245 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:35.952294 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:35.953669 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:35.953675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:35.955146 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:35.955163 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:35.955976 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:35.956001 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:35.956267 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:35.957058 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:35.957083 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:35.959617 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:35.963266 (+ 3649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:35.963276 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:35.963297 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:35.963744 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:35.964127 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:35.964133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:35.964196 (+ 63us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:35.964455 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:35.965142 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:35.965151 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:35.965692 (+ 541us) client_negotiation.cc:769] Sending connection context
0806 01:13:35.966062 (+ 370us) client_negotiation.cc:240] Negotiation successful
0806 01:13:35.966793 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":564,"threads_started":1}
I20260806 01:13:35.969333 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:35.945319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:38459 (local address 127.25.236.190:33751)
0806 01:13:35.945940 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:35.945948 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:35.947077 (+ 1129us) server_negotiation.cc:407] Connection header received
0806 01:13:35.950207 (+ 3130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:35.950211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:35.950356 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:35.950505 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:35.952546 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:35.953449 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:35.955318 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:35.955738 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:35.955838 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:35.959867 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:35.959872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:35.959876 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:35.959910 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:35.963108 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:35.963877 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:35.963887 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:35.963889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:35.964016 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:35.964604 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:35.964609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:35.964612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:35.964951 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:35.965146 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:35.968845 (+ 3699us) server_negotiation.cc:299] Negotiation successful
0806 01:13:35.969054 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":272,"threads_started":1}
I20260806 01:13:35.970732 28077 heartbeater.cc:347] Connected to a master server at 127.25.236.190:33751
I20260806 01:13:35.971297 28077 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:35.972420 28077 heartbeater.cc:511] Master 127.25.236.190:33751 requested a full tablet report, sending...
I20260806 01:13:35.974560 27389 ts_manager.cc:194] Registered new tserver with Master: 26a9ac3402a742db8b1b0d64ece7b4ca (127.25.236.131:46851)
I20260806 01:13:35.975400 27389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:38459
I20260806 01:13:35.987807 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:13:36.012923 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:35.996855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36092 (local address 127.25.236.190:33751)
0806 01:13:35.997106 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:35.997113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:35.997691 (+ 578us) server_negotiation.cc:407] Connection header received
0806 01:13:35.997940 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:35.997944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:35.998052 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:35.998158 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:35.999812 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.000523 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.002148 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.002648 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.002790 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.006032 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:36.006037 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:36.006041 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:36.006078 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:36.009818 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.010762 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.010768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.010776 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.010911 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.011309 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.011320 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.011323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.011681 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:36.011968 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.012320 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.012580 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:13:36.022750 27389 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:13:36.031160 27389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:13:36.067130 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.050732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:33223 (local address 127.0.0.1:48398)
0806 01:13:36.051986 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:36.052062 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:36.052385 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:36.055596 (+ 3211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:36.055607 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:36.055658 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:36.056345 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.056368 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.059408 (+ 3040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.059417 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.063277 (+ 3860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.063300 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.065524 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.065531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.065766 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.065831 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 01:13:36.065944 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 01:13:36.066399 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"thread_start_us":636,"threads_started":1}
I20260806 01:13:36.068009 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.050898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.25.236.129:33223)
0806 01:13:36.054749 (+ 3851us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:36.054757 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:36.054785 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:13:36.054902 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:36.054913 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:36.055287 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:36.056679 (+ 1392us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:36.057003 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.059180 (+ 2177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.063602 (+ 4422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.065280 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.066804 (+ 1524us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.067183 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.067290 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.067505 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3681,"thread_start_us":3521,"threads_started":1}
I20260806 01:13:36.069331 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.050955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:41081 (local address 127.0.0.1:55482)
0806 01:13:36.054618 (+ 3663us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:36.054645 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:36.054888 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:36.056044 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:36.056052 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:36.056093 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:36.058710 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.058732 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.063269 (+ 4537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.063278 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.066544 (+ 3266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.066565 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.068474 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.068480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.068658 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.068695 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:13:36.068796 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 01:13:36.068942 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3516,"thread_start_us":1840,"threads_started":1}
I20260806 01:13:36.069378 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.052139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55482 (local address 127.25.236.130:41081)
0806 01:13:36.054146 (+ 2007us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:36.054160 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:36.054763 (+ 603us) server_negotiation.cc:407] Connection header received
0806 01:13:36.055168 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:36.055190 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:36.055483 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:36.055684 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:36.059035 (+ 3351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.062987 (+ 3952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.066767 (+ 3780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.068272 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.068448 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.068619 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.068867 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.069025 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1728,"thread_start_us":2493,"threads_started":1}
I20260806 01:13:36.073606 27575 tablet_service.cc:1467] Processing CreateTablet for tablet 0b8ed5ddab0d4ca3a643b4c14b8c9e85 (DEFAULT_TABLE table=test-table [id=9709dc629f194e12a92e89ec49903b0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:36.073606 27774 tablet_service.cc:1467] Processing CreateTablet for tablet 0b8ed5ddab0d4ca3a643b4c14b8c9e85 (DEFAULT_TABLE table=test-table [id=9709dc629f194e12a92e89ec49903b0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:36.075085 27575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b8ed5ddab0d4ca3a643b4c14b8c9e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:36.075162 27774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b8ed5ddab0d4ca3a643b4c14b8c9e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:36.079252 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.053885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:46851 (local address 127.0.0.1:59728)
0806 01:13:36.064379 (+ 10494us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:36.064405 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:36.064627 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:36.071533 (+ 6906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:36.071537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:36.071568 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:36.071944 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.071957 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.074043 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.074048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.076266 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.076281 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.078407 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.078414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.078590 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.078628 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 01:13:36.078733 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 01:13:36.078877 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10351,"thread_start_us":10844,"threads_started":1}
I20260806 01:13:36.079557 28104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.054743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59728 (local address 127.25.236.131:46851)
0806 01:13:36.070386 (+ 15643us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:36.070395 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:36.070430 (+ 35us) server_negotiation.cc:407] Connection header received
0806 01:13:36.070543 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:36.070552 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:36.070850 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:36.071063 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:36.072238 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.073870 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.076444 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.078096 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.078311 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.078534 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.078792 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.078961 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15434,"thread_start_us":15206,"threads_started":1}
I20260806 01:13:36.083678 27975 tablet_service.cc:1467] Processing CreateTablet for tablet 0b8ed5ddab0d4ca3a643b4c14b8c9e85 (DEFAULT_TABLE table=test-table [id=9709dc629f194e12a92e89ec49903b0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:36.085497 27975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b8ed5ddab0d4ca3a643b4c14b8c9e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:36.089191 28106 tablet_bootstrap.cc:492] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf: Bootstrap starting.
I20260806 01:13:36.091406 28107 tablet_bootstrap.cc:492] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b: Bootstrap starting.
I20260806 01:13:36.094208 28106 tablet_bootstrap.cc:654] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:36.095810 28107 tablet_bootstrap.cc:654] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:36.095896 28106 log.cc:824] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:36.097111 28107 log.cc:824] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:36.099411 28106 tablet_bootstrap.cc:492] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf: No bootstrap required, opened a new log
I20260806 01:13:36.099735 28106 ts_tablet_manager.cc:1397] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.001s
I20260806 01:13:36.100261 28107 tablet_bootstrap.cc:492] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b: No bootstrap required, opened a new log
I20260806 01:13:36.100528 28107 ts_tablet_manager.cc:1397] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 01:13:36.100739 28111 tablet_bootstrap.cc:492] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca: Bootstrap starting.
I20260806 01:13:36.103729 28111 tablet_bootstrap.cc:654] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:36.105504 28111 log.cc:824] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:36.108139 28111 tablet_bootstrap.cc:492] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca: No bootstrap required, opened a new log
I20260806 01:13:36.108394 28111 ts_tablet_manager.cc:1397] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260806 01:13:36.111548 28106 raft_consensus.cc:348] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33223 } } peers { permanent_uuid: "545d82876e78442080403f933b3566bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41081 } } peers { permanent_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46851 } }
I20260806 01:13:36.111548 28107 raft_consensus.cc:348] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33223 } } peers { permanent_uuid: "545d82876e78442080403f933b3566bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41081 } } peers { permanent_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46851 } }
I20260806 01:13:36.112025 28107 raft_consensus.cc:374] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:36.112025 28106 raft_consensus.cc:374] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:36.112150 28106 raft_consensus.cc:729] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545d82876e78442080403f933b3566bf, State: Initialized, Role: FOLLOWER
I20260806 01:13:36.112139 28107 raft_consensus.cc:729] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a62dc3a7121b43fb9af81c15cf308d8b, State: Initialized, Role: FOLLOWER
I20260806 01:13:36.112821 28107 consensus_queue.cc:260] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33223 } } peers { permanent_uuid: "545d82876e78442080403f933b3566bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41081 } } peers { permanent_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46851 } }
I20260806 01:13:36.112821 28106 consensus_queue.cc:260] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33223 } } peers { permanent_uuid: "545d82876e78442080403f933b3566bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41081 } } peers { permanent_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46851 } }
I20260806 01:13:36.114655 27876 heartbeater.cc:503] Master 127.25.236.190:33751 was elected leader, sending a full tablet report...
I20260806 01:13:36.114899 28107 ts_tablet_manager.cc:1428] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260806 01:13:36.117727 28106 ts_tablet_manager.cc:1428] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf: Time spent starting tablet: real 0.018s user 0.012s sys 0.003s
I20260806 01:13:36.118505 28111 raft_consensus.cc:348] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33223 } } peers { permanent_uuid: "545d82876e78442080403f933b3566bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41081 } } peers { permanent_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46851 } }
I20260806 01:13:36.119050 28111 raft_consensus.cc:374] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:36.119182 28111 raft_consensus.cc:729] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26a9ac3402a742db8b1b0d64ece7b4ca, State: Initialized, Role: FOLLOWER
I20260806 01:13:36.119966 28111 consensus_queue.cc:260] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33223 } } peers { permanent_uuid: "545d82876e78442080403f933b3566bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41081 } } peers { permanent_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46851 } }
I20260806 01:13:36.121323 28077 heartbeater.cc:503] Master 127.25.236.190:33751 was elected leader, sending a full tablet report...
I20260806 01:13:36.122066 28111 ts_tablet_manager.cc:1428] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca: Time spent starting tablet: real 0.014s user 0.011s sys 0.001s
W20260806 01:13:36.193979 28079 tablet.cc:2369] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:13:36.232365 27676 tablet.cc:2369] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:13:36.284569 27877 tablet.cc:2369] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:13:36.429144 28116 raft_consensus.cc:482] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:13:36.429618 28116 raft_consensus.cc:504] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33223 } } peers { permanent_uuid: "545d82876e78442080403f933b3566bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41081 } } peers { permanent_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46851 } }
I20260806 01:13:36.432044 28116 leader_election.cc:288] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a62dc3a7121b43fb9af81c15cf308d8b (127.25.236.129:33223), 26a9ac3402a742db8b1b0d64ece7b4ca (127.25.236.131:46851)
I20260806 01:13:36.435256 28113 raft_consensus.cc:482] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:13:36.435622 28113 raft_consensus.cc:504] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33223 } } peers { permanent_uuid: "545d82876e78442080403f933b3566bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41081 } } peers { permanent_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46851 } }
I20260806 01:13:36.437490 28113 leader_election.cc:288] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 545d82876e78442080403f933b3566bf (127.25.236.130:41081), 26a9ac3402a742db8b1b0d64ece7b4ca (127.25.236.131:46851)
I20260806 01:13:36.445784 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.432624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:46851 (local address 127.25.236.130:60929)
0806 01:13:36.433647 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:36.433667 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:36.433884 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:36.434370 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:36.434373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:36.434397 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:36.434870 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.434883 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.436277 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.436285 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.438670 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.438691 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.440993 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.441002 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.442571 (+ 1569us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.442640 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 01:13:36.444601 (+ 1961us) client_negotiation.cc:240] Negotiation successful
0806 01:13:36.444902 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":38,"thread_start_us":594,"threads_started":1}
I20260806 01:13:36.444044 28104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.432580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:60929 (local address 127.25.236.131:46851)
0806 01:13:36.432810 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:36.432819 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:36.433781 (+ 962us) server_negotiation.cc:407] Connection header received
0806 01:13:36.434063 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:36.434069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:36.434155 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:36.434297 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:36.435066 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.436127 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.438896 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.440494 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.440653 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.440757 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.442879 (+ 2122us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.443719 (+ 840us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:13:36.447184 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.432559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:51995 (local address 127.25.236.129:33223)
0806 01:13:36.432814 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:36.432821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:36.433619 (+ 798us) server_negotiation.cc:407] Connection header received
0806 01:13:36.433809 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:36.433816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:36.433899 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:36.433999 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:36.434685 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.435654 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.444488 (+ 8834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.445751 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.445863 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.445933 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.446805 (+ 872us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.446951 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:13:36.447456 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.437950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:37799 (local address 127.25.236.130:41081)
0806 01:13:36.438176 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:36.438185 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:36.438887 (+ 702us) server_negotiation.cc:407] Connection header received
0806 01:13:36.439159 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:36.439165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:36.439260 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:36.439330 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:36.440163 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.441579 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.444314 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.445670 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.445806 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.446108 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.446910 (+ 802us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.447032 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":65}
I20260806 01:13:36.448002 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.432366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:33223 (local address 127.25.236.130:51995)
0806 01:13:36.433478 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:36.433499 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:36.433652 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:36.434033 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:36.434043 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:36.434071 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:36.434522 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.434534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.435796 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.435804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.443534 (+ 7730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.444327 (+ 793us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.446415 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.446423 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.446599 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.446641 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 01:13:36.447429 (+ 788us) client_negotiation.cc:240] Negotiation successful
0806 01:13:36.447603 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"mutex_wait_us":5,"thread_start_us":654,"threads_started":1}
I20260806 01:13:36.449335 27606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8ed5ddab0d4ca3a643b4c14b8c9e85" candidate_uuid: "545d82876e78442080403f933b3566bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" is_pre_election: true
I20260806 01:13:36.449436 28007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8ed5ddab0d4ca3a643b4c14b8c9e85" candidate_uuid: "545d82876e78442080403f933b3566bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca" is_pre_election: true
I20260806 01:13:36.450220 28007 raft_consensus.cc:2454] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 545d82876e78442080403f933b3566bf in term 0.
I20260806 01:13:36.451032 27711 leader_election.cc:302] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26a9ac3402a742db8b1b0d64ece7b4ca, 545d82876e78442080403f933b3566bf; no voters:
I20260806 01:13:36.451524 28116 raft_consensus.cc:2781] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:13:36.451450 27606 raft_consensus.cc:2454] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 545d82876e78442080403f933b3566bf in term 0.
I20260806 01:13:36.451617 28121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.437867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:41081 (local address 127.25.236.129:37799)
0806 01:13:36.438784 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:36.438804 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:36.439000 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:36.439527 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:36.439531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:36.439566 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:36.439986 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.439999 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.441773 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.441781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.444108 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.444124 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.446340 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.446350 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.446730 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.446776 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 01:13:36.447903 (+ 1127us) client_negotiation.cc:240] Negotiation successful
0806 01:13:36.449797 (+ 1894us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":564,"threads_started":1}
I20260806 01:13:36.451704 28116 raft_consensus.cc:482] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:13:36.451812 28116 raft_consensus.cc:3037] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:36.453049 27802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8ed5ddab0d4ca3a643b4c14b8c9e85" candidate_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "545d82876e78442080403f933b3566bf" is_pre_election: true
I20260806 01:13:36.455000 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.439508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:54273 (local address 127.25.236.131:46851)
0806 01:13:36.442544 (+ 3036us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:36.442550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:36.442577 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:13:36.442640 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:36.442645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:36.442723 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:36.442825 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:36.443947 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.448246 (+ 4299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.452952 (+ 4706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.453900 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.454096 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.454172 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.454495 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.454634 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2901,"thread_start_us":2650,"threads_started":1}
I20260806 01:13:36.455004 28123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.440030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:46851 (local address 127.25.236.129:54273)
0806 01:13:36.441652 (+ 1622us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:36.441682 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:36.441847 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:36.443238 (+ 1391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:36.443244 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:36.443274 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:36.443749 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.443763 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.448654 (+ 4891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.448660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.452785 (+ 4125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:36.452799 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.454115 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.454122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:36.454331 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.454356 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 01:13:36.454458 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 01:13:36.454595 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1485,"thread_start_us":1300,"threads_started":1}
I20260806 01:13:36.454946 28116 raft_consensus.cc:504] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33223 } } peers { permanent_uuid: "545d82876e78442080403f933b3566bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41081 } } peers { permanent_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46851 } }
I20260806 01:13:36.455657 27802 raft_consensus.cc:2379] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a62dc3a7121b43fb9af81c15cf308d8b in current term 1: Already voted for candidate 545d82876e78442080403f933b3566bf in this term.
I20260806 01:13:36.455688 28007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8ed5ddab0d4ca3a643b4c14b8c9e85" candidate_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca" is_pre_election: true
I20260806 01:13:36.455936 28007 raft_consensus.cc:2454] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a62dc3a7121b43fb9af81c15cf308d8b in term 0.
I20260806 01:13:36.456444 27509 leader_election.cc:302] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26a9ac3402a742db8b1b0d64ece7b4ca, a62dc3a7121b43fb9af81c15cf308d8b; no voters:
I20260806 01:13:36.456525 28116 leader_election.cc:288] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [CANDIDATE]: Term 1 election: Requested vote from peers a62dc3a7121b43fb9af81c15cf308d8b (127.25.236.129:33223), 26a9ac3402a742db8b1b0d64ece7b4ca (127.25.236.131:46851)
I20260806 01:13:36.456573 28007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8ed5ddab0d4ca3a643b4c14b8c9e85" candidate_uuid: "545d82876e78442080403f933b3566bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca"
I20260806 01:13:36.456792 28007 raft_consensus.cc:3037] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:36.457209 27606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8ed5ddab0d4ca3a643b4c14b8c9e85" candidate_uuid: "545d82876e78442080403f933b3566bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a62dc3a7121b43fb9af81c15cf308d8b"
I20260806 01:13:36.457485 27606 raft_consensus.cc:3037] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:36.459697 28007 raft_consensus.cc:2454] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 26a9ac3402a742db8b1b0d64ece7b4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 545d82876e78442080403f933b3566bf in term 1.
I20260806 01:13:36.460124 27711 leader_election.cc:302] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26a9ac3402a742db8b1b0d64ece7b4ca, 545d82876e78442080403f933b3566bf; no voters:
I20260806 01:13:36.460455 28116 raft_consensus.cc:2781] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:36.460917 27606 raft_consensus.cc:2454] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 545d82876e78442080403f933b3566bf in term 1.
I20260806 01:13:36.461017 28116 raft_consensus.cc:686] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [term 1 LEADER]: Becoming Leader. State: Replica: 545d82876e78442080403f933b3566bf, State: Running, Role: LEADER
I20260806 01:13:36.461097 28113 raft_consensus.cc:2741] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P a62dc3a7121b43fb9af81c15cf308d8b [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 01:13:36.461678 28116 consensus_queue.cc:237] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33223 } } peers { permanent_uuid: "545d82876e78442080403f933b3566bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41081 } } peers { permanent_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46851 } }
I20260806 01:13:36.468528 27387 catalog_manager.cc:5495] T 0b8ed5ddab0d4ca3a643b4c14b8c9e85 P 545d82876e78442080403f933b3566bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 545d82876e78442080403f933b3566bf (127.25.236.130). New cstate: current_term: 1 leader_uuid: "545d82876e78442080403f933b3566bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62dc3a7121b43fb9af81c15cf308d8b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "545d82876e78442080403f933b3566bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26a9ac3402a742db8b1b0d64ece7b4ca" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46851 } health_report { overall_health: UNKNOWN } } }
Aug 06 01:13:36 dist-test-slave-7nm7 krb5kdc[27330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:13:36.529306 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.512816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48408 (local address 127.25.236.129:33223)
0806 01:13:36.513003 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:36.513008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:36.513616 (+ 608us) server_negotiation.cc:407] Connection header received
0806 01:13:36.513880 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:36.513887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:36.513964 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:36.514074 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:36.515465 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.516217 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.517458 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.517877 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.518015 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.522296 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:36.522320 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:36.522338 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:36.522380 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:36.525974 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.526985 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.526998 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.527000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.527231 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.527652 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.527661 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.527665 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.528059 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:36.528385 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.528715 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.528951 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 01:13:36 dist-test-slave-7nm7 krb5kdc[27330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978816, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:13:36 dist-test-slave-7nm7 krb5kdc[27330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978816, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:13:36.584496 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.569198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48412 (local address 127.25.236.129:33223)
0806 01:13:36.569414 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:36.569423 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:36.569780 (+ 357us) server_negotiation.cc:407] Connection header received
0806 01:13:36.570009 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:36.570019 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:36.570080 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:36.570378 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:36.572093 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.572876 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.574067 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.574812 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.574951 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.578979 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:36.578986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:36.578994 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:36.579028 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:36.581936 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.582659 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.582663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.582665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.582788 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.583194 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.583204 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.583206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.583506 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:36.583736 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.584069 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.584253 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 01:13:36 dist-test-slave-7nm7 krb5kdc[27330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:13:36 dist-test-slave-7nm7 krb5kdc[27330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:13:36.644925 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.628030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.25.236.129:33223)
0806 01:13:36.628488 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:36.628496 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:36.628785 (+ 289us) server_negotiation.cc:407] Connection header received
0806 01:13:36.628905 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:36.628910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:36.628978 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:36.629070 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:36.630782 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.631687 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.633091 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.633599 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.633703 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.638420 (+ 4717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:36.638426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:36.638429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:36.638466 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:36.641446 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.642294 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.642304 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.642311 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.642462 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.643027 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.643034 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.643036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.643587 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:36.644028 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.644213 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.644477 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
W20260806 01:13:36.647812 27544 tablet_service.cc:2908] Rejecting scan request for tablet 0b8ed5ddab0d4ca3a643b4c14b8c9e85: Uninitialized: clean time has not yet been initialized
I20260806 01:13:36.652565 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27482
I20260806 01:13:36.668195 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.653581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.25.236.129:33223)
0806 01:13:36.653853 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:36.653862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:36.654096 (+ 234us) server_negotiation.cc:407] Connection header received
0806 01:13:36.654376 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:36.654380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:36.654458 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:36.654570 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:36.656082 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.656803 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.658048 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.658536 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.658687 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.661596 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:36.661601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:36.661605 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:36.661640 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:36.664463 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.665474 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.665482 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.665488 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.665725 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.666258 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.666267 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.666364 (+ 97us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.666785 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:36.667138 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.667519 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.667792 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 01:13:36.669003 27666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:36.770650 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 27482
I20260806 01:13:36.802709 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27683
Aug 06 01:13:36 dist-test-slave-7nm7 krb5kdc[27330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:13:36.818714 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.803497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55498 (local address 127.25.236.130:41081)
0806 01:13:36.803678 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:36.803683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:36.803704 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:13:36.803953 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:36.803958 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:36.804026 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:36.804112 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:36.805457 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.806201 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.807755 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.808169 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.808263 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.812669 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:36.812687 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:36.812703 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:36.812749 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:36.815724 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.816420 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.816427 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.816429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.816583 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.816951 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.816955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.816957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.817476 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:36.817713 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.818079 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.818389 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:13:36.819522 27869 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:36.918198 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 27683
I20260806 01:13:36.943594 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27884
Aug 06 01:13:36 dist-test-slave-7nm7 krb5kdc[27330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:13:36.963871 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:36.944407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59730 (local address 127.25.236.131:46851)
0806 01:13:36.944591 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:36.944595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:36.944671 (+ 76us) server_negotiation.cc:407] Connection header received
0806 01:13:36.944891 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:36.944895 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:36.944962 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:36.945055 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:36.947307 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.948405 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.951297 (+ 2892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:36.951761 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:36.951874 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:36.957237 (+ 5363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:36.957251 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:36.957265 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:36.957313 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:36.960900 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.961703 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.961710 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.961712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.961838 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:36.962266 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:36.962274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:36.962277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:36.962684 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:36.963004 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:36.963291 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 01:13:36.963591 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:13:36.964537 28067 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:37.060623 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 27884
I20260806 01:13:37.085942 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27347
I20260806 01:13:37.087456 27443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:37.209830 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 27347
2026-08-06T01:13:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (5278 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:13:37 dist-test-slave-7nm7 krb5kdc[28148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:13:37 dist-test-slave-7nm7 krb5kdc[28148](info): set up 2 sockets
Aug 06 01:13:37 dist-test-slave-7nm7 krb5kdc[28148](info): commencing operation
krb5kdc: starting...
W20260806 01:13:38.586210 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:13:38 dist-test-slave-7nm7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:13:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:13:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:13:38.814426 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:40135
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:34481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:40135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:39.189618 28167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:39.189936 28167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:39.190133 28167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:39.197832 28167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:39.197949 28167 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 01:13:39.198060 28167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:39.198199 28167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:39.198294 28167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:39.198401 28167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:39.208685 28167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:34481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:40135
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:40135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:39.210073 28167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:39.211447 28167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:39.223740 28175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:39.223698 28178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:39.223786 28176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:39.227895 28167 server_base.cc:1034] running on GCE node
I20260806 01:13:39.228734 28167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:39.230192 28167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:39.231483 28167 hybrid_clock.cc:648] HybridClock initialized: now 1785978819231423 us; error 32 us; skew 500 ppm
I20260806 01:13:39.231823 28167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:39 dist-test-slave-7nm7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978819, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:39.241268 28167 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:13:39.243049 28167 webserver.cc:460] Webserver started at http://127.25.236.190:33859/ using document root <none> and password file <none>
I20260806 01:13:39.243832 28167 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:39.243963 28167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:39.244324 28167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:39.247467 28167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "b8eabf5f59644edd9ea73c5ad2a25858"
format_stamp: "Formatted at 2026-08-06 01:13:39 on dist-test-slave-7nm7"
server_key: "2ca37e6104092d408f1c4542be3675a3"
server_key_iv: "33ee6e3c5989ed013bba231d3fe3c1dd"
server_key_version: "encryptionkey@0"
I20260806 01:13:39.248260 28167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "b8eabf5f59644edd9ea73c5ad2a25858"
format_stamp: "Formatted at 2026-08-06 01:13:39 on dist-test-slave-7nm7"
server_key: "2ca37e6104092d408f1c4542be3675a3"
server_key_iv: "33ee6e3c5989ed013bba231d3fe3c1dd"
server_key_version: "encryptionkey@0"
I20260806 01:13:39.253620 28167 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:13:39.257191 28191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:39.258600 28167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:13:39.258790 28167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "b8eabf5f59644edd9ea73c5ad2a25858"
format_stamp: "Formatted at 2026-08-06 01:13:39 on dist-test-slave-7nm7"
server_key: "2ca37e6104092d408f1c4542be3675a3"
server_key_iv: "33ee6e3c5989ed013bba231d3fe3c1dd"
server_key_version: "encryptionkey@0"
I20260806 01:13:39.259023 28167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:39.275516 28167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:39.279860 28167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:39.280158 28167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:39.315780 28167 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:40135
I20260806 01:13:39.315804 28273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:40135 every 8 connection(s)
I20260806 01:13:39.317288 28167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:13:39.318528 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 28167
I20260806 01:13:39.318797 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:13:39.319600 26546 external_mini_cluster.cc:1428] Setting key 0689544b2e23076aa5366f68941c5f89
I20260806 01:13:39.322497 28274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:13:39 dist-test-slave-7nm7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:39.337577 28274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858: Bootstrap starting.
I20260806 01:13:39.342096 28274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:39.343781 28274 log.cc:824] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:39.348196 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:39.322224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40382 (local address 127.25.236.190:40135)
0806 01:13:39.323252 (+ 1028us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:39.323269 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:39.323325 (+ 56us) server_negotiation.cc:407] Connection header received
0806 01:13:39.325816 (+ 2491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:39.325846 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:39.326523 (+ 677us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:39.327033 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:39.328382 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:39.330702 (+ 2320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:39.331985 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:39.333009 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:39.333193 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:39.337726 (+ 4533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:39.337739 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:39.337758 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:39.337811 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:39.341911 (+ 4100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:39.342877 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:39.342886 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:39.342896 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:39.343116 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:39.343597 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:39.343603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:39.343606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:39.344703 (+ 1097us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:39.345084 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:39.345987 (+ 903us) server_negotiation.cc:299] Negotiation successful
0806 01:13:39.346509 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":436,"threads_started":1}
I20260806 01:13:39.348410 28274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858: No bootstrap required, opened a new log
I20260806 01:13:39.356462 28274 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8eabf5f59644edd9ea73c5ad2a25858" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40135 } }
I20260806 01:13:39.356980 28274 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:39.357097 28274 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8eabf5f59644edd9ea73c5ad2a25858, State: Initialized, Role: FOLLOWER
I20260806 01:13:39.357756 28274 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8eabf5f59644edd9ea73c5ad2a25858" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40135 } }
I20260806 01:13:39.358014 28274 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:13:39.358171 28274 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:13:39.358412 28274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:39.360652 28274 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8eabf5f59644edd9ea73c5ad2a25858" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40135 } }
I20260806 01:13:39.361207 28274 leader_election.cc:302] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8eabf5f59644edd9ea73c5ad2a25858; no voters:
I20260806 01:13:39.361837 28274 leader_election.cc:288] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:13:39.361956 28280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:39.364323 28280 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [term 1 LEADER]: Becoming Leader. State: Replica: b8eabf5f59644edd9ea73c5ad2a25858, State: Running, Role: LEADER
I20260806 01:13:39.364902 28274 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:39.364954 28280 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8eabf5f59644edd9ea73c5ad2a25858" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40135 } }
I20260806 01:13:39.368422 28283 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8eabf5f59644edd9ea73c5ad2a25858. Latest consensus state: current_term: 1 leader_uuid: "b8eabf5f59644edd9ea73c5ad2a25858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8eabf5f59644edd9ea73c5ad2a25858" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40135 } } }
I20260806 01:13:39.368587 28282 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8eabf5f59644edd9ea73c5ad2a25858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8eabf5f59644edd9ea73c5ad2a25858" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40135 } } }
I20260806 01:13:39.368875 28283 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:39.368875 28282 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:39.370647 28293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:13:39.375939 28293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:13:39.386386 28293 catalog_manager.cc:1347] Generated new cluster ID: 161b006a9d3949f7a9be6957c72ed4d6
I20260806 01:13:39.386531 28293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:13:39.414494 28293 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:13:39.416801 28293 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:13:39.431543 28293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858: Generated new TSK 0
I20260806 01:13:39.432724 28293 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:13:39.498927 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:40135
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:34481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:39.874953 28307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:39.875324 28307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:39.875622 28307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:39.884616 28307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:39.884812 28307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:39.884964 28307 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:13:39.885109 28307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:13:39.896652 28307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:34481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.236.190:40135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:39.898126 28307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:39.899948 28307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:39.913970 28313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:39.913901 28312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:39.913988 28317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:39.914862 28307 server_base.cc:1034] running on GCE node
I20260806 01:13:39.915645 28307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:39.916621 28307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:39.917891 28307 hybrid_clock.cc:648] HybridClock initialized: now 1785978819917818 us; error 35 us; skew 500 ppm
I20260806 01:13:39.918423 28307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:39 dist-test-slave-7nm7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978819, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:39.921841 28307 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:13:39.923669 28307 webserver.cc:460] Webserver started at http://127.25.236.129:37803/ using document root <none> and password file <none>
I20260806 01:13:39.924381 28307 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:39.924474 28307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:39.924757 28307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:39.927819 28307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "c09dd495f2554c9baeacc7c0a5c89b11"
format_stamp: "Formatted at 2026-08-06 01:13:39 on dist-test-slave-7nm7"
server_key: "a2f3a028f76cb5842cacdf611aa1044d"
server_key_iv: "76c7513635bd609ef39c442ab5eaa8b0"
server_key_version: "encryptionkey@0"
I20260806 01:13:39.928589 28307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "c09dd495f2554c9baeacc7c0a5c89b11"
format_stamp: "Formatted at 2026-08-06 01:13:39 on dist-test-slave-7nm7"
server_key: "a2f3a028f76cb5842cacdf611aa1044d"
server_key_iv: "76c7513635bd609ef39c442ab5eaa8b0"
server_key_version: "encryptionkey@0"
I20260806 01:13:39.934413 28307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:13:39.938117 28326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:39.939873 28307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:13:39.940069 28307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "c09dd495f2554c9baeacc7c0a5c89b11"
format_stamp: "Formatted at 2026-08-06 01:13:39 on dist-test-slave-7nm7"
server_key: "a2f3a028f76cb5842cacdf611aa1044d"
server_key_iv: "76c7513635bd609ef39c442ab5eaa8b0"
server_key_version: "encryptionkey@0"
I20260806 01:13:39.940433 28307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:39.959657 28307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:39.964326 28307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:39.964609 28307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:39.967938 28307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:39.968142 28307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 01:13:39.968324 28307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:39.968444 28307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:13:39 dist-test-slave-7nm7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978819, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:39.996515 28337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:39.968642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:44189 (local address 127.25.236.190:40135)
0806 01:13:39.971218 (+ 2576us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:39.971226 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:39.971253 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:13:39.972757 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:39.972764 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:39.972852 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:39.972980 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:39.975157 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:39.976159 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:39.980095 (+ 3936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:39.980604 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:39.980762 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:39.987243 (+ 6481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:39.987248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:39.987251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:39.987309 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:39.991318 (+ 4009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:39.992761 (+ 1443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:39.992770 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:39.992773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:39.993622 (+ 849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:39.994363 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:39.994368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:39.994372 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:39.994797 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:39.994969 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:39.995872 (+ 903us) server_negotiation.cc:299] Negotiation successful
0806 01:13:39.996124 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2342,"thread_start_us":2193,"threads_started":1}
I20260806 01:13:39.997622 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:39.969225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:40135 (local address 127.25.236.129:44189)
0806 01:13:39.970330 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:39.970390 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:39.972458 (+ 2068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:39.973244 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:39.973256 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:39.974042 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:39.974937 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:39.974998 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:39.976360 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:39.976367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:39.979866 (+ 3499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:39.979889 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:39.980901 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:39.980911 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:39.981186 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:39.982147 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:39.982177 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:39.986884 (+ 4707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:39.991558 (+ 4674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:39.991571 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:39.991592 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:39.992551 (+ 959us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:39.993790 (+ 1239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:39.993801 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:39.993808 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:39.994177 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:39.994946 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:39.994955 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:39.995440 (+ 485us) client_negotiation.cc:769] Sending connection context
0806 01:13:39.995740 (+ 300us) client_negotiation.cc:240] Negotiation successful
0806 01:13:39.996333 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":410,"threads_started":1}
I20260806 01:13:40.019754 28337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:40.004499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:52729 (local address 127.25.236.190:40135)
0806 01:13:40.004715 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:40.004723 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:40.005422 (+ 699us) server_negotiation.cc:407] Connection header received
0806 01:13:40.005620 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:40.005629 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:40.005735 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:40.005872 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:40.007161 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:40.008263 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:40.010513 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:40.010961 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:40.011094 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:40.013054 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:40.013062 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:40.013072 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:40.013104 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:40.016412 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:40.017188 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:40.017197 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:40.017201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:40.017340 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:40.017820 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:40.017829 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:40.017835 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:40.018348 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:40.018538 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:40.019109 (+ 571us) server_negotiation.cc:299] Negotiation successful
0806 01:13:40.019358 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:13:40.019891 28438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:40.004593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:40135 (local address 127.25.236.129:52729)
0806 01:13:40.005301 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:40.005329 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:40.005455 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:40.005891 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:40.005895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:40.006420 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:40.006915 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:40.006934 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:40.008427 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:40.008434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:40.010347 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:40.010360 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:40.011199 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:40.011206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:40.011367 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:40.012058 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:40.012076 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:40.012849 (+ 773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:40.016569 (+ 3720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:40.016574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:40.016579 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:40.017039 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:40.017461 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:40.017470 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:40.017474 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:40.017668 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:40.018502 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:40.018507 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:40.018779 (+ 272us) client_negotiation.cc:769] Sending connection context
0806 01:13:40.018988 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 01:13:40.019378 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":381,"threads_started":1}
I20260806 01:13:40.031562 28307 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:35425
I20260806 01:13:40.031562 28500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:35425 every 8 connection(s)
I20260806 01:13:40.032778 28307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:13:40.034921 28501 heartbeater.cc:347] Connected to a master server at 127.25.236.190:40135
I20260806 01:13:40.035166 28501 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:40.035842 28501 heartbeater.cc:511] Master 127.25.236.190:40135 requested a full tablet report, sending...
I20260806 01:13:40.036556 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 28307
I20260806 01:13:40.036814 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:13:40.037748 26546 external_mini_cluster.cc:1428] Setting key 88d98a02dd469fae0686f54b308b2e67
I20260806 01:13:40.038504 28215 ts_manager.cc:194] Registered new tserver with Master: c09dd495f2554c9baeacc7c0a5c89b11 (127.25.236.129:35425)
I20260806 01:13:40.040678 28215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:44189
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:13:40.086107 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:40135
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:34481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:40.468715 28507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:40.469002 28507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:40.469219 28507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:40.477361 28507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:40.477629 28507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:40.477943 28507 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:13:40.478183 28507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:13:40.488561 28507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:34481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.236.190:40135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:40.489945 28507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:40.491353 28507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:40.505106 28515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:40.505069 28521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:40.505113 28518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:40.506111 28507 server_base.cc:1034] running on GCE node
I20260806 01:13:40.506783 28507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:40.507711 28507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:40.508992 28507 hybrid_clock.cc:648] HybridClock initialized: now 1785978820508897 us; error 36 us; skew 500 ppm
I20260806 01:13:40.509341 28507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:40 dist-test-slave-7nm7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978820, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:40.512928 28507 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:13:40.514588 28507 webserver.cc:460] Webserver started at http://127.25.236.130:33047/ using document root <none> and password file <none>
I20260806 01:13:40.515324 28507 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:40.515448 28507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:40.515797 28507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:40.518903 28507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "7c43b5bb8e5e4843b89b60dcc0505096"
format_stamp: "Formatted at 2026-08-06 01:13:40 on dist-test-slave-7nm7"
server_key: "aed6fa51c0563fddd98cb57cd7929cc1"
server_key_iv: "0e251a2ae566d37b2cc46e91a97fe7f1"
server_key_version: "encryptionkey@0"
I20260806 01:13:40.519837 28507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "7c43b5bb8e5e4843b89b60dcc0505096"
format_stamp: "Formatted at 2026-08-06 01:13:40 on dist-test-slave-7nm7"
server_key: "aed6fa51c0563fddd98cb57cd7929cc1"
server_key_iv: "0e251a2ae566d37b2cc46e91a97fe7f1"
server_key_version: "encryptionkey@0"
I20260806 01:13:40.525867 28507 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 01:13:40.529659 28528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:40.531164 28507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:13:40.531369 28507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "7c43b5bb8e5e4843b89b60dcc0505096"
format_stamp: "Formatted at 2026-08-06 01:13:40 on dist-test-slave-7nm7"
server_key: "aed6fa51c0563fddd98cb57cd7929cc1"
server_key_iv: "0e251a2ae566d37b2cc46e91a97fe7f1"
server_key_version: "encryptionkey@0"
I20260806 01:13:40.531605 28507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:40.554183 28507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:40.559495 28507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:40.559829 28507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:40.562968 28507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:40.563046 28507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:40.563264 28507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:40.563364 28507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:13:40 dist-test-slave-7nm7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978820, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:40.590044 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:40.563879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:43163 (local address 127.25.236.190:40135)
0806 01:13:40.564394 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:40.564403 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:40.565757 (+ 1354us) server_negotiation.cc:407] Connection header received
0806 01:13:40.567853 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:40.567861 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:40.567997 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:40.568134 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:40.571138 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:40.572522 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:40.575965 (+ 3443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:40.576502 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:40.576677 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:40.581753 (+ 5076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:40.581758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:40.581761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:40.581795 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:40.585879 (+ 4084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:40.587012 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:40.587018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:40.587021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:40.587187 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:40.587958 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:40.587964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:40.587967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:40.588340 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:40.588467 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:40.589575 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0806 01:13:40.589754 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":257,"threads_started":1}
I20260806 01:13:40.592350 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:40.564606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:40135 (local address 127.25.236.130:43163)
0806 01:13:40.565535 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:40.565580 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:40.567565 (+ 1985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:40.568652 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:40.568666 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:40.569648 (+ 982us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:40.570841 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:40.570884 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:40.572758 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:40.572766 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:40.575696 (+ 2930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:40.575734 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:40.576840 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:40.576852 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:40.577248 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:40.578295 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:40.578345 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:40.581499 (+ 3154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:40.586056 (+ 4557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:40.586068 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:40.586089 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:40.586823 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:40.587345 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:40.587357 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:40.587367 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:40.587777 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:40.588484 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:40.588498 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:40.589095 (+ 597us) client_negotiation.cc:769] Sending connection context
0806 01:13:40.589497 (+ 402us) client_negotiation.cc:240] Negotiation successful
0806 01:13:40.590259 (+ 762us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":441,"threads_started":1}
I20260806 01:13:40.616806 28507 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:38945
I20260806 01:13:40.616832 28700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:38945 every 8 connection(s)
I20260806 01:13:40.618148 28507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:13:40.621191 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 28507
I20260806 01:13:40.621354 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:13:40.621407 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:40.605391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:60089 (local address 127.25.236.190:40135)
0806 01:13:40.605632 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:40.605640 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:40.606435 (+ 795us) server_negotiation.cc:407] Connection header received
0806 01:13:40.606716 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:40.606726 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:40.606825 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:40.606908 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:40.608708 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:40.609793 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:40.612478 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:40.612894 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:40.613056 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:40.615176 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:40.615188 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:40.615192 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:40.615223 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:40.618215 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:40.619040 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:40.619045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:40.619049 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:40.619171 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:40.619550 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:40.619556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:40.619559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:40.619897 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:40.620063 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:40.620828 (+ 765us) server_negotiation.cc:299] Negotiation successful
0806 01:13:40.621068 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 01:13:40.621451 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:40.605330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:40135 (local address 127.25.236.130:60089)
0806 01:13:40.606296 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:40.606320 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:40.606524 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:40.607139 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:40.607142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:40.607913 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:40.608522 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:40.608548 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:40.609942 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:40.609947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:40.612318 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:40.612331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:40.613098 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:40.613104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:40.613268 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:40.613940 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:40.613958 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:40.614965 (+ 1007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:40.618422 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:40.618433 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:40.618437 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:40.618884 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:40.619287 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:40.619292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:40.619296 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:40.619429 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:40.620304 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:40.620309 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:40.620507 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 01:13:40.620657 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 01:13:40.620987 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":473,"threads_started":1}
I20260806 01:13:40.621910 26546 external_mini_cluster.cc:1428] Setting key 84fcd07bea7c15f7f3a69f56fdb8b6eb
I20260806 01:13:40.622311 28701 heartbeater.cc:347] Connected to a master server at 127.25.236.190:40135
I20260806 01:13:40.622577 28701 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:40.623430 28701 heartbeater.cc:511] Master 127.25.236.190:40135 requested a full tablet report, sending...
I20260806 01:13:40.625089 28215 ts_manager.cc:194] Registered new tserver with Master: 7c43b5bb8e5e4843b89b60dcc0505096 (127.25.236.130:38945)
I20260806 01:13:40.626057 28215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:43163
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:13:40.677300 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:40135
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:34481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:13:41.043889 28501 heartbeater.cc:503] Master 127.25.236.190:40135 was elected leader, sending a full tablet report...
W20260806 01:13:41.058557 28707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:41.058843 28707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:41.058950 28707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:41.066540 28707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:41.066711 28707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:41.066855 28707 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:13:41.066998 28707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:13:41.076876 28707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:34481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.236.190:40135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:41.078167 28707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:41.079551 28707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:41.092602 28721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:41.092559 28717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:41.092646 28716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:41.095794 28707 server_base.cc:1034] running on GCE node
I20260806 01:13:41.096434 28707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:41.097424 28707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:41.098680 28707 hybrid_clock.cc:648] HybridClock initialized: now 1785978821098607 us; error 47 us; skew 500 ppm
I20260806 01:13:41.099009 28707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:41 dist-test-slave-7nm7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978821, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:41.102643 28707 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:13:41.104525 28707 webserver.cc:460] Webserver started at http://127.25.236.131:39135/ using document root <none> and password file <none>
I20260806 01:13:41.105341 28707 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:41.105484 28707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:41.105845 28707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:41.108974 28707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "5df60cc1933144209fe334035c7e262b"
format_stamp: "Formatted at 2026-08-06 01:13:41 on dist-test-slave-7nm7"
server_key: "a370247713c53405f99aac88454b5d2f"
server_key_iv: "f43ca1e7209a33c5daf9e56e24e307fa"
server_key_version: "encryptionkey@0"
I20260806 01:13:41.109824 28707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "5df60cc1933144209fe334035c7e262b"
format_stamp: "Formatted at 2026-08-06 01:13:41 on dist-test-slave-7nm7"
server_key: "a370247713c53405f99aac88454b5d2f"
server_key_iv: "f43ca1e7209a33c5daf9e56e24e307fa"
server_key_version: "encryptionkey@0"
I20260806 01:13:41.116045 28707 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260806 01:13:41.119979 28729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:41.121805 28707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:13:41.121997 28707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "5df60cc1933144209fe334035c7e262b"
format_stamp: "Formatted at 2026-08-06 01:13:41 on dist-test-slave-7nm7"
server_key: "a370247713c53405f99aac88454b5d2f"
server_key_iv: "f43ca1e7209a33c5daf9e56e24e307fa"
server_key_version: "encryptionkey@0"
I20260806 01:13:41.122265 28707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:41.151877 28707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:41.157313 28707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:41.157814 28707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:41.162093 28707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:41.162259 28707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:41.162442 28707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:41.162513 28707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:13:41 dist-test-slave-7nm7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978821, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:41.189544 28739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.162466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:46943 (local address 127.25.236.190:40135)
0806 01:13:41.162945 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:41.162953 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:41.164557 (+ 1604us) server_negotiation.cc:407] Connection header received
0806 01:13:41.167123 (+ 2566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:41.167129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:41.167346 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:41.167490 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:41.170515 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.171645 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.174735 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.175502 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.175678 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.182020 (+ 6342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:41.182025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:41.182028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:41.182065 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:41.185919 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:41.187053 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:41.187063 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:41.187066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:41.187198 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:41.187664 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:41.187676 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:41.187678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:41.187950 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:41.188111 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:41.188889 (+ 778us) server_negotiation.cc:299] Negotiation successful
0806 01:13:41.189113 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":236,"threads_started":1}
I20260806 01:13:41.191666 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.163344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:40135 (local address 127.25.236.131:46943)
0806 01:13:41.164387 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:41.164436 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:41.166821 (+ 2385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:41.167831 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:41.167840 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:41.169247 (+ 1407us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:41.170319 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.170342 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.171816 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.171824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.174195 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.174217 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.175801 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.175810 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.176056 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.177200 (+ 1144us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:41.177225 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:41.181734 (+ 4509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:41.186093 (+ 4359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:41.186104 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:41.186123 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:41.186901 (+ 778us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:41.187319 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:41.187324 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:41.187327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:41.187538 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:41.188094 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:41.188102 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:41.188499 (+ 397us) client_negotiation.cc:769] Sending connection context
0806 01:13:41.188780 (+ 281us) client_negotiation.cc:240] Negotiation successful
0806 01:13:41.189344 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":508,"threads_started":1}
I20260806 01:13:41.221024 28707 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:44933
I20260806 01:13:41.221037 28907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:44933 every 8 connection(s)
I20260806 01:13:41.222853 28707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:13:41.223179 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 28707
I20260806 01:13:41.223387 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:13:41.224049 26546 external_mini_cluster.cc:1428] Setting key 895a0e5d39ef1e2fd3b086a26f617705
I20260806 01:13:41.225533 28908 heartbeater.cc:347] Connected to a master server at 127.25.236.190:40135
I20260806 01:13:41.225835 28908 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:41.226948 28908 heartbeater.cc:511] Master 127.25.236.190:40135 requested a full tablet report, sending...
I20260806 01:13:41.228582 28214 ts_manager.cc:194] Registered new tserver with Master: 5df60cc1933144209fe334035c7e262b (127.25.236.131:44933)
I20260806 01:13:41.229827 28875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.209522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:40135 (local address 127.25.236.131:39529)
0806 01:13:41.210760 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:41.210782 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:41.210973 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:41.211425 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:41.211433 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:41.212748 (+ 1315us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:41.213997 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.214031 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.215784 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.215792 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.218125 (+ 2333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.218148 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.219049 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.219059 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.219545 (+ 486us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.220805 (+ 1260us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:41.220837 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:41.222082 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:41.226500 (+ 4418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:41.226511 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:41.226517 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:41.227124 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:41.227641 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:41.227645 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:41.227649 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:41.227806 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:41.228612 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:41.228617 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:41.228825 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 01:13:41.228982 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 01:13:41.229349 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"thread_start_us":959,"threads_started":1}
I20260806 01:13:41.229909 28214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:46943
I20260806 01:13:41.230463 28739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.209508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:39529 (local address 127.25.236.190:40135)
0806 01:13:41.209734 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:41.209743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:41.210895 (+ 1152us) server_negotiation.cc:407] Connection header received
0806 01:13:41.211129 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:41.211137 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:41.211236 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:41.211348 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:41.214304 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.215567 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.218348 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.218786 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.218945 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.222504 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:41.222510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:41.222513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:41.222550 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:41.226327 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:41.227297 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:41.227303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:41.227306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:41.227478 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:41.227945 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:41.227950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:41.227955 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:41.228369 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:41.228501 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:41.229202 (+ 701us) server_negotiation.cc:299] Negotiation successful
0806 01:13:41.229540 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:13:41.240829 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:13:41 dist-test-slave-7nm7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978821, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:13:41 dist-test-slave-7nm7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978821, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:41.303381 28739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.286326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40388 (local address 127.25.236.190:40135)
0806 01:13:41.286513 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:41.286517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:41.286787 (+ 270us) server_negotiation.cc:407] Connection header received
0806 01:13:41.287036 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:41.287042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:41.287140 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:41.287246 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:41.288710 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.289485 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.291581 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.292025 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.292146 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.297298 (+ 5152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:41.297303 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:41.297306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:41.297341 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:41.300131 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:41.300966 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:41.300971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:41.300973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:41.301100 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:41.301517 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:41.301520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:41.301522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:41.301806 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:41.302007 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:41.302621 (+ 614us) server_negotiation.cc:299] Negotiation successful
0806 01:13:41.303085 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:13:41.308511 28214 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:13:41.318207 28214 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:13:41.350674 28930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.337055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50924 (local address 127.25.236.131:44933)
0806 01:13:41.339745 (+ 2690us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:41.339754 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:41.339789 (+ 35us) server_negotiation.cc:407] Connection header received
0806 01:13:41.339890 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:41.339901 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:41.340190 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:41.341420 (+ 1230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:41.341843 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.343662 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.347632 (+ 3970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.349388 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.349623 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.349837 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:41.350100 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 01:13:41.350308 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2508,"thread_start_us":1798,"threads_started":1}
I20260806 01:13:41.350682 28929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.336642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:44933 (local address 127.0.0.1:50924)
0806 01:13:41.337778 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:41.337834 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:41.338080 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:41.340507 (+ 2427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:41.340537 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:41.340581 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:41.341164 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.341183 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.343933 (+ 2750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.343943 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.347398 (+ 3455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.347414 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.349615 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.349621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.349829 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.349894 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 01:13:41.350053 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 01:13:41.350307 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":667,"threads_started":1}
I20260806 01:13:41.355105 28807 tablet_service.cc:1467] Processing CreateTablet for tablet 4164c53da1de4ca9bcad10235e138734 (DEFAULT_TABLE table=test-table [id=d5274bd7dcbc4bf4afe5d5ca4f980cb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:41.356601 28935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.338044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:35425 (local address 127.0.0.1:47474)
0806 01:13:41.347215 (+ 9171us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:41.347236 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:41.347395 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:41.348321 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:41.348325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:41.348345 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:41.348608 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.348614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.350997 (+ 2383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.351003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.354011 (+ 3008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.354027 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.355720 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.355727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.355890 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.355930 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 01:13:41.356057 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 01:13:41.356196 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8332,"mutex_wait_us":207,"thread_start_us":8251,"threads_started":1}
I20260806 01:13:41.357163 28807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4164c53da1de4ca9bcad10235e138734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:41.357426 28934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.338813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47474 (local address 127.25.236.129:35425)
0806 01:13:41.345856 (+ 7043us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:41.345949 (+ 93us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:41.347534 (+ 1585us) server_negotiation.cc:407] Connection header received
0806 01:13:41.347624 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:41.347630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:41.347927 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:41.348129 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:41.348778 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.350719 (+ 1941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.354272 (+ 3553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.355469 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.355708 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.355880 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:41.356923 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0806 01:13:41.357130 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6828,"thread_start_us":7685,"threads_started":1}
I20260806 01:13:41.360318 28931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.338461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:38945 (local address 127.0.0.1:49804)
0806 01:13:41.347301 (+ 8840us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:41.347320 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:41.347403 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:41.350687 (+ 3284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:41.350693 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:41.350733 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:41.351901 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.351921 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.353976 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.353981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.356914 (+ 2933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.356933 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.359226 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.359233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.359509 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.359563 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 01:13:41.359698 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 01:13:41.359844 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8694,"thread_start_us":8560,"threads_started":1}
I20260806 01:13:41.361225 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.338705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49804 (local address 127.25.236.130:38945)
0806 01:13:41.349194 (+ 10489us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:41.349202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:41.349229 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:13:41.349552 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:41.349566 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:41.349972 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:41.350357 (+ 385us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:41.352235 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.353795 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.357104 (+ 3309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.358979 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.359199 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.359499 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:41.360784 (+ 1285us) server_negotiation.cc:299] Negotiation successful
0806 01:13:41.360982 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10317,"thread_start_us":10184,"threads_started":1}
I20260806 01:13:41.361286 28402 tablet_service.cc:1467] Processing CreateTablet for tablet 4164c53da1de4ca9bcad10235e138734 (DEFAULT_TABLE table=test-table [id=d5274bd7dcbc4bf4afe5d5ca4f980cb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:41.362833 28402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4164c53da1de4ca9bcad10235e138734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:41.364363 28602 tablet_service.cc:1467] Processing CreateTablet for tablet 4164c53da1de4ca9bcad10235e138734 (DEFAULT_TABLE table=test-table [id=d5274bd7dcbc4bf4afe5d5ca4f980cb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:41.365558 28602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4164c53da1de4ca9bcad10235e138734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:41.374984 28938 tablet_bootstrap.cc:492] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b: Bootstrap starting.
I20260806 01:13:41.379709 28944 tablet_bootstrap.cc:492] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096: Bootstrap starting.
I20260806 01:13:41.380352 28938 tablet_bootstrap.cc:654] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:41.380782 28949 tablet_bootstrap.cc:492] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11: Bootstrap starting.
I20260806 01:13:41.382395 28938 log.cc:824] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:41.385370 28938 tablet_bootstrap.cc:492] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b: No bootstrap required, opened a new log
I20260806 01:13:41.385372 28944 tablet_bootstrap.cc:654] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:41.385377 28949 tablet_bootstrap.cc:654] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:41.385673 28938 ts_tablet_manager.cc:1397] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 01:13:41.386828 28944 log.cc:824] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:41.386829 28949 log.cc:824] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:41.393842 28949 tablet_bootstrap.cc:492] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11: No bootstrap required, opened a new log
I20260806 01:13:41.394392 28949 ts_tablet_manager.cc:1397] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260806 01:13:41.395134 28944 tablet_bootstrap.cc:492] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096: No bootstrap required, opened a new log
I20260806 01:13:41.395486 28944 ts_tablet_manager.cc:1397] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260806 01:13:41.395838 28938 raft_consensus.cc:348] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } }
I20260806 01:13:41.396646 28938 raft_consensus.cc:374] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:41.396782 28938 raft_consensus.cc:729] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df60cc1933144209fe334035c7e262b, State: Initialized, Role: FOLLOWER
I20260806 01:13:41.397716 28938 consensus_queue.cc:260] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } }
I20260806 01:13:41.399329 28908 heartbeater.cc:503] Master 127.25.236.190:40135 was elected leader, sending a full tablet report...
I20260806 01:13:41.399886 28938 ts_tablet_manager.cc:1428] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b: Time spent starting tablet: real 0.014s user 0.012s sys 0.003s
I20260806 01:13:41.406343 28949 raft_consensus.cc:348] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } }
I20260806 01:13:41.406517 28944 raft_consensus.cc:348] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } }
I20260806 01:13:41.406893 28949 raft_consensus.cc:374] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:41.406994 28944 raft_consensus.cc:374] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:41.407009 28949 raft_consensus.cc:729] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c09dd495f2554c9baeacc7c0a5c89b11, State: Initialized, Role: FOLLOWER
I20260806 01:13:41.407125 28944 raft_consensus.cc:729] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c43b5bb8e5e4843b89b60dcc0505096, State: Initialized, Role: FOLLOWER
I20260806 01:13:41.407683 28949 consensus_queue.cc:260] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } }
I20260806 01:13:41.407728 28944 consensus_queue.cc:260] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } }
I20260806 01:13:41.409032 28701 heartbeater.cc:503] Master 127.25.236.190:40135 was elected leader, sending a full tablet report...
I20260806 01:13:41.409610 28944 ts_tablet_manager.cc:1428] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260806 01:13:41.410135 28949 ts_tablet_manager.cc:1428] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
W20260806 01:13:41.484351 28909 tablet.cc:2369] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:13:41.535598 28502 tablet.cc:2369] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:13:41.621929 28702 tablet.cc:2369] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:13:41.815186 28955 raft_consensus.cc:482] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:13:41.815593 28955 raft_consensus.cc:504] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } }
I20260806 01:13:41.817345 28955 leader_election.cc:288] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c43b5bb8e5e4843b89b60dcc0505096 (127.25.236.130:38945), c09dd495f2554c9baeacc7c0a5c89b11 (127.25.236.129:35425)
I20260806 01:13:41.827454 28934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.817929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:58145 (local address 127.25.236.129:35425)
0806 01:13:41.818475 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:41.818480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:41.818650 (+ 170us) server_negotiation.cc:407] Connection header received
0806 01:13:41.818933 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:41.818939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:41.819014 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:41.819137 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:41.819830 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.820924 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.824457 (+ 3533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.826009 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.826288 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.826357 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:41.826944 (+ 587us) server_negotiation.cc:299] Negotiation successful
0806 01:13:41.827176 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380}
I20260806 01:13:41.827455 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.817902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:36653 (local address 127.25.236.130:38945)
0806 01:13:41.818274 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:41.818284 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:41.818842 (+ 558us) server_negotiation.cc:407] Connection header received
0806 01:13:41.818929 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:41.818934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:41.819015 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:41.819129 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:41.819761 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.820936 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.824476 (+ 3540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.826006 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.826275 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.826357 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:41.826943 (+ 586us) server_negotiation.cc:299] Negotiation successful
0806 01:13:41.827183 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260806 01:13:41.827618 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.817704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:38945 (local address 127.25.236.131:36653)
0806 01:13:41.818703 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:41.818719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:41.818826 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:41.819157 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:41.819162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:41.819189 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:41.819516 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.819532 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.821273 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.821283 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.824123 (+ 2840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.824141 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.826390 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.826401 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.826774 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.826802 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 01:13:41.826894 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 01:13:41.827091 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":204,"spinlock_wait_cycles":18304,"thread_start_us":395,"threads_started":1}
I20260806 01:13:41.827904 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.817679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:35425 (local address 127.25.236.131:58145)
0806 01:13:41.818522 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:41.818544 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:41.818712 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:41.819157 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:41.819162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:41.819278 (+ 116us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:41.819652 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.819663 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.821275 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.821283 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.824120 (+ 2837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:41.824141 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.826425 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.826430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:41.826659 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.826781 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 01:13:41.826894 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 01:13:41.827091 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":247,"spinlock_wait_cycles":27136,"thread_start_us":738,"threads_started":1}
I20260806 01:13:41.828984 28635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4164c53da1de4ca9bcad10235e138734" candidate_uuid: "5df60cc1933144209fe334035c7e262b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" is_pre_election: true
I20260806 01:13:41.829286 28434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4164c53da1de4ca9bcad10235e138734" candidate_uuid: "5df60cc1933144209fe334035c7e262b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" is_pre_election: true
I20260806 01:13:41.829938 28434 raft_consensus.cc:2454] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5df60cc1933144209fe334035c7e262b in term 0.
I20260806 01:13:41.830034 28635 raft_consensus.cc:2454] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5df60cc1933144209fe334035c7e262b in term 0.
I20260806 01:13:41.830796 28731 leader_election.cc:302] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5df60cc1933144209fe334035c7e262b, 7c43b5bb8e5e4843b89b60dcc0505096; no voters:
I20260806 01:13:41.831249 28955 raft_consensus.cc:2781] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:13:41.831534 28955 raft_consensus.cc:482] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:13:41.831682 28955 raft_consensus.cc:3037] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:41.833959 28955 raft_consensus.cc:504] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } }
I20260806 01:13:41.834648 28955 leader_election.cc:288] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [CANDIDATE]: Term 1 election: Requested vote from peers 7c43b5bb8e5e4843b89b60dcc0505096 (127.25.236.130:38945), c09dd495f2554c9baeacc7c0a5c89b11 (127.25.236.129:35425)
I20260806 01:13:41.835134 28434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4164c53da1de4ca9bcad10235e138734" candidate_uuid: "5df60cc1933144209fe334035c7e262b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c09dd495f2554c9baeacc7c0a5c89b11"
I20260806 01:13:41.835125 28635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4164c53da1de4ca9bcad10235e138734" candidate_uuid: "5df60cc1933144209fe334035c7e262b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c43b5bb8e5e4843b89b60dcc0505096"
I20260806 01:13:41.835362 28635 raft_consensus.cc:3037] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:41.835430 28434 raft_consensus.cc:3037] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:41.837775 28635 raft_consensus.cc:2454] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5df60cc1933144209fe334035c7e262b in term 1.
I20260806 01:13:41.837779 28434 raft_consensus.cc:2454] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5df60cc1933144209fe334035c7e262b in term 1.
I20260806 01:13:41.838387 28730 leader_election.cc:302] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5df60cc1933144209fe334035c7e262b, c09dd495f2554c9baeacc7c0a5c89b11; no voters:
I20260806 01:13:41.839077 28955 raft_consensus.cc:2781] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:41.839543 28955 raft_consensus.cc:686] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [term 1 LEADER]: Becoming Leader. State: Replica: 5df60cc1933144209fe334035c7e262b, State: Running, Role: LEADER
I20260806 01:13:41.840097 28955 consensus_queue.cc:237] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } }
I20260806 01:13:41.846421 28214 catalog_manager.cc:5495] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5df60cc1933144209fe334035c7e262b (127.25.236.131). New cstate: current_term: 1 leader_uuid: "5df60cc1933144209fe334035c7e262b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260806 01:13:41.935832 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:41.928475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50938 (local address 127.25.236.131:44933)
0806 01:13:41.928992 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:41.928997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:41.929102 (+ 105us) server_negotiation.cc:407] Connection header received
0806 01:13:41.929499 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:41.929506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:41.929628 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:41.929860 (+ 232us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:41.930493 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.931225 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.932901 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:41.933437 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:41.933543 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:41.934178 (+ 635us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:41.935188 (+ 1010us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:41.935305 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:41.935513 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 01:13:41.935594 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":304,"threads_started":1}
I20260806 01:13:41.945659 28434 raft_consensus.cc:1261] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11 [term 1 FOLLOWER]: Refusing update from remote peer 5df60cc1933144209fe334035c7e262b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:41.946938 28955 consensus_queue.cc:1035] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:41.948096 28635 raft_consensus.cc:1261] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 1 FOLLOWER]: Refusing update from remote peer 5df60cc1933144209fe334035c7e262b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:41.949054 28955 consensus_queue.cc:1035] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:41.958879 28978 mvcc.cc:205] Tried to move back new op lower bound from 7315369254675726336 to 7315369254263758848. Current Snapshot: MvccSnapshot[applied={T|T < 7315369254675726336}]
I20260806 01:13:41.984261 28215 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:13:41.984709 28215 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:13:41.991231 28215 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858: Sending DeleteTablet for 3 replicas of tablet 4164c53da1de4ca9bcad10235e138734
I20260806 01:13:41.993193 28602 tablet_service.cc:1514] Processing DeleteTablet for tablet 4164c53da1de4ca9bcad10235e138734 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:41 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:49804
I20260806 01:13:41.993454 28807 tablet_service.cc:1514] Processing DeleteTablet for tablet 4164c53da1de4ca9bcad10235e138734 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:41 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:50924
I20260806 01:13:41.993419 28402 tablet_service.cc:1514] Processing DeleteTablet for tablet 4164c53da1de4ca9bcad10235e138734 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:41 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:47474
I20260806 01:13:42.001708 28214 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:13:42.002914 28214 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:13:42.004196 28990 tablet_replica.cc:330] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096: stopping tablet replica
I20260806 01:13:42.005201 28990 raft_consensus.cc:2229] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:13:42.014225 28401 tablet_service.cc:1467] Processing CreateTablet for tablet 39822454beee4b6c9b8f869a97c8556b (DEFAULT_TABLE table=test-table [id=e75fa24d9bec41fea5bf108cdde983df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:42.014596 28805 tablet_service.cc:1467] Processing CreateTablet for tablet 39822454beee4b6c9b8f869a97c8556b (DEFAULT_TABLE table=test-table [id=e75fa24d9bec41fea5bf108cdde983df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:42.015108 28401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39822454beee4b6c9b8f869a97c8556b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:42.014487 28602 tablet_service.cc:1467] Processing CreateTablet for tablet 39822454beee4b6c9b8f869a97c8556b (DEFAULT_TABLE table=test-table [id=e75fa24d9bec41fea5bf108cdde983df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:42.015738 28805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39822454beee4b6c9b8f869a97c8556b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:42.016302 28602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39822454beee4b6c9b8f869a97c8556b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:42.023501 28992 tablet_replica.cc:330] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11: stopping tablet replica
I20260806 01:13:42.024068 28992 raft_consensus.cc:2229] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:13:42.025066 28992 raft_consensus.cc:2258] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:42.025689 28990 raft_consensus.cc:2258] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:42.029304 28990 ts_tablet_manager.cc:1905] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:42.039821 28994 tablet_bootstrap.cc:492] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11: Bootstrap starting.
I20260806 01:13:42.042845 28994 tablet_bootstrap.cc:654] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:42.043979 28991 tablet_replica.cc:330] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b: stopping tablet replica
I20260806 01:13:42.045354 28991 raft_consensus.cc:2229] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:13:42.046033 28992 ts_tablet_manager.cc:1905] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:42.046341 28991 raft_consensus.cc:2258] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:42.051154 28997 tablet_bootstrap.cc:492] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096: Bootstrap starting.
I20260806 01:13:42.053444 28997 tablet_bootstrap.cc:654] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:42.056378 28991 ts_tablet_manager.cc:1905] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:42.061720 28992 ts_tablet_manager.cc:1918] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:13:42.062033 28992 log.cc:1196] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/4164c53da1de4ca9bcad10235e138734
I20260806 01:13:42.065160 28993 tablet_bootstrap.cc:492] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b: Bootstrap starting.
I20260806 01:13:42.066660 28990 ts_tablet_manager.cc:1918] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:13:42.066906 28990 log.cc:1196] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/4164c53da1de4ca9bcad10235e138734
I20260806 01:13:42.067438 28990 ts_tablet_manager.cc:1939] T 4164c53da1de4ca9bcad10235e138734 P 7c43b5bb8e5e4843b89b60dcc0505096: Deleting consensus metadata
I20260806 01:13:42.071175 28992 ts_tablet_manager.cc:1939] T 4164c53da1de4ca9bcad10235e138734 P c09dd495f2554c9baeacc7c0a5c89b11: Deleting consensus metadata
I20260806 01:13:42.069996 28198 catalog_manager.cc:4841] TS 7c43b5bb8e5e4843b89b60dcc0505096 (127.25.236.130:38945): tablet 4164c53da1de4ca9bcad10235e138734 (table test-table [id=d5274bd7dcbc4bf4afe5d5ca4f980cb5]) successfully deleted
I20260806 01:13:42.072503 28997 tablet_bootstrap.cc:492] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096: No bootstrap required, opened a new log
I20260806 01:13:42.072705 28997 ts_tablet_manager.cc:1397] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.004s
I20260806 01:13:42.073792 28997 raft_consensus.cc:348] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.074066 28997 raft_consensus.cc:374] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:42.074247 28997 raft_consensus.cc:729] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c43b5bb8e5e4843b89b60dcc0505096, State: Initialized, Role: FOLLOWER
I20260806 01:13:42.074724 28997 consensus_queue.cc:260] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.075529 28192 catalog_manager.cc:4841] TS c09dd495f2554c9baeacc7c0a5c89b11 (127.25.236.129:35425): tablet 4164c53da1de4ca9bcad10235e138734 (table test-table [id=d5274bd7dcbc4bf4afe5d5ca4f980cb5]) successfully deleted
I20260806 01:13:42.076594 28997 ts_tablet_manager.cc:1428] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260806 01:13:42.080035 28994 tablet_bootstrap.cc:492] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11: No bootstrap required, opened a new log
I20260806 01:13:42.080323 28994 ts_tablet_manager.cc:1397] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11: Time spent bootstrapping tablet: real 0.041s user 0.007s sys 0.006s
I20260806 01:13:42.080389 28993 tablet_bootstrap.cc:654] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:42.081688 28994 raft_consensus.cc:348] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.083739 28994 raft_consensus.cc:374] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:42.083921 28994 raft_consensus.cc:729] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c09dd495f2554c9baeacc7c0a5c89b11, State: Initialized, Role: FOLLOWER
I20260806 01:13:42.084259 28994 consensus_queue.cc:260] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.084833 28991 ts_tablet_manager.cc:1918] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:13:42.085161 28991 log.cc:1196] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/4164c53da1de4ca9bcad10235e138734
I20260806 01:13:42.085729 28994 ts_tablet_manager.cc:1428] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260806 01:13:42.088332 28991 ts_tablet_manager.cc:1939] T 4164c53da1de4ca9bcad10235e138734 P 5df60cc1933144209fe334035c7e262b: Deleting consensus metadata
I20260806 01:13:42.089351 28993 tablet_bootstrap.cc:492] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b: No bootstrap required, opened a new log
I20260806 01:13:42.089531 28993 ts_tablet_manager.cc:1397] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.008s
I20260806 01:13:42.090487 28993 raft_consensus.cc:348] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.090656 28198 catalog_manager.cc:4841] TS 5df60cc1933144209fe334035c7e262b (127.25.236.131:44933): tablet 4164c53da1de4ca9bcad10235e138734 (table test-table [id=d5274bd7dcbc4bf4afe5d5ca4f980cb5]) successfully deleted
I20260806 01:13:42.090777 28993 raft_consensus.cc:374] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:42.090883 28993 raft_consensus.cc:729] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df60cc1933144209fe334035c7e262b, State: Initialized, Role: FOLLOWER
I20260806 01:13:42.091166 28993 consensus_queue.cc:260] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.091938 28993 ts_tablet_manager.cc:1428] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 01:13:42.328668 28970 raft_consensus.cc:482] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:13:42.328930 28970 raft_consensus.cc:504] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.331496 28970 leader_election.cc:288] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c43b5bb8e5e4843b89b60dcc0505096 (127.25.236.130:38945), 5df60cc1933144209fe334035c7e262b (127.25.236.131:44933)
I20260806 01:13:42.349740 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.331799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:38945 (local address 127.25.236.129:59357)
0806 01:13:42.339033 (+ 7234us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:42.339053 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:42.339253 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:42.340293 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:42.340299 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:42.340355 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:42.340853 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:42.340868 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.342799 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.342808 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:42.345883 (+ 3075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:42.345901 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.348446 (+ 2545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.348453 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:42.348661 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.348713 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 01:13:42.348832 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 01:13:42.348995 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7113,"mutex_wait_us":1,"thread_start_us":6999,"threads_started":1}
I20260806 01:13:42.349792 29001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.331918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:59357 (local address 127.25.236.130:38945)
0806 01:13:42.338800 (+ 6882us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:42.338809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:42.339171 (+ 362us) server_negotiation.cc:407] Connection header received
0806 01:13:42.339767 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:42.339774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:42.339866 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:42.340004 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:42.341040 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.342648 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.346183 (+ 3535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.348192 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.348411 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.348487 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:42.348970 (+ 483us) server_negotiation.cc:299] Negotiation successful
0806 01:13:42.349267 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6713,"thread_start_us":6602,"threads_started":1}
I20260806 01:13:42.350085 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.339573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:47321 (local address 127.25.236.131:44933)
0806 01:13:42.339778 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:42.339784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:42.340689 (+ 905us) server_negotiation.cc:407] Connection header received
0806 01:13:42.340903 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:42.340909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:42.340993 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:42.341096 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:42.342219 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.343787 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.347166 (+ 3379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.348565 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.348794 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.348876 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:42.349624 (+ 748us) server_negotiation.cc:299] Negotiation successful
0806 01:13:42.349764 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:13:42.350826 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.339485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:44933 (local address 127.25.236.129:47321)
0806 01:13:42.340546 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:42.340565 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:42.340735 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:42.341279 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:42.341284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:42.341323 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:42.341902 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:42.341918 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.343969 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.343975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:42.346927 (+ 2952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:42.346940 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.348889 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.348897 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:42.349293 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.349438 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 01:13:42.350171 (+ 733us) client_negotiation.cc:240] Negotiation successful
0806 01:13:42.350379 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"thread_start_us":603,"threads_started":1}
I20260806 01:13:42.351651 28635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39822454beee4b6c9b8f869a97c8556b" candidate_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" is_pre_election: true
I20260806 01:13:42.351926 28635 raft_consensus.cc:2454] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c09dd495f2554c9baeacc7c0a5c89b11 in term 0.
I20260806 01:13:42.352072 28843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39822454beee4b6c9b8f869a97c8556b" candidate_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df60cc1933144209fe334035c7e262b" is_pre_election: true
I20260806 01:13:42.352550 28330 leader_election.cc:302] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c43b5bb8e5e4843b89b60dcc0505096, c09dd495f2554c9baeacc7c0a5c89b11; no voters:
I20260806 01:13:42.352625 28843 raft_consensus.cc:2454] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c09dd495f2554c9baeacc7c0a5c89b11 in term 0.
I20260806 01:13:42.353066 28970 raft_consensus.cc:2781] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:13:42.353286 28970 raft_consensus.cc:482] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:13:42.353433 28970 raft_consensus.cc:3037] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:42.356400 28970 raft_consensus.cc:504] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.357110 28970 leader_election.cc:288] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [CANDIDATE]: Term 1 election: Requested vote from peers 7c43b5bb8e5e4843b89b60dcc0505096 (127.25.236.130:38945), 5df60cc1933144209fe334035c7e262b (127.25.236.131:44933)
I20260806 01:13:42.357712 28635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39822454beee4b6c9b8f869a97c8556b" candidate_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c43b5bb8e5e4843b89b60dcc0505096"
I20260806 01:13:42.357796 28843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39822454beee4b6c9b8f869a97c8556b" candidate_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df60cc1933144209fe334035c7e262b"
I20260806 01:13:42.358022 28843 raft_consensus.cc:3037] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:42.358052 28635 raft_consensus.cc:3037] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:42.360875 28843 raft_consensus.cc:2454] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c09dd495f2554c9baeacc7c0a5c89b11 in term 1.
I20260806 01:13:42.361616 28330 leader_election.cc:302] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5df60cc1933144209fe334035c7e262b, c09dd495f2554c9baeacc7c0a5c89b11; no voters:
I20260806 01:13:42.362006 28970 raft_consensus.cc:2781] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:42.362742 28635 raft_consensus.cc:2454] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c09dd495f2554c9baeacc7c0a5c89b11 in term 1.
I20260806 01:13:42.362950 28970 raft_consensus.cc:686] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [term 1 LEADER]: Becoming Leader. State: Replica: c09dd495f2554c9baeacc7c0a5c89b11, State: Running, Role: LEADER
I20260806 01:13:42.363615 28970 consensus_queue.cc:237] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.370012 28215 catalog_manager.cc:5495] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 reported cstate change: term changed from 0 to 1, leader changed from <none> to c09dd495f2554c9baeacc7c0a5c89b11 (127.25.236.129). New cstate: current_term: 1 leader_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } health_report { overall_health: UNKNOWN } } }
Aug 06 01:13:42 dist-test-slave-7nm7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978819, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:42.467092 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.447598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:40135 (local address 127.0.0.1:40392)
0806 01:13:42.450637 (+ 3039us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:42.450680 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:42.450884 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:42.451230 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:42.451234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:42.452384 (+ 1150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:42.452699 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:42.452710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.453964 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.453970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:42.455317 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:42.455330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.456267 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.456275 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:42.456535 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.457281 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:42.457299 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:42.459801 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:42.463790 (+ 3989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:42.463805 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:42.463813 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:42.464258 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:42.464691 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:42.464695 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:42.464699 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:42.464850 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:42.465672 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:42.465681 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:42.465862 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 01:13:42.466108 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 01:13:42.466451 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2874,"thread_start_us":2696,"threads_started":1}
I20260806 01:13:42.467180 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.447615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.25.236.190:40135)
0806 01:13:42.450132 (+ 2517us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:42.450141 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:42.450773 (+ 632us) server_negotiation.cc:407] Connection header received
0806 01:13:42.451013 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:42.451018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:42.451086 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:42.451200 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:42.452865 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.453813 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.455459 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.456046 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.456188 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.460048 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:42.460053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:42.460056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:42.460084 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:42.463661 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:42.464384 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:42.464396 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:42.464398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:42.464567 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:42.464965 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:42.464970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:42.464973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:42.465529 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:42.465708 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:42.466209 (+ 501us) server_negotiation.cc:299] Negotiation successful
0806 01:13:42.466596 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":134,"server-negotiator.queue_time_us":2242,"thread_start_us":2187,"threads_started":1}
I20260806 01:13:42.474215 28215 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:40392:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 01:13:42.475865 28215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:13:42.483837 28401 tablet_service.cc:1467] Processing CreateTablet for tablet 742725f311d64b53b1eddd4cabaab3e0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4d7e530d69bd4705a3ba0a9b0578706b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:13:42.484133 28602 tablet_service.cc:1467] Processing CreateTablet for tablet 742725f311d64b53b1eddd4cabaab3e0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4d7e530d69bd4705a3ba0a9b0578706b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:13:42.484447 28401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 742725f311d64b53b1eddd4cabaab3e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:42.484772 28602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 742725f311d64b53b1eddd4cabaab3e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:42.487437 28805 tablet_service.cc:1467] Processing CreateTablet for tablet 742725f311d64b53b1eddd4cabaab3e0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4d7e530d69bd4705a3ba0a9b0578706b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:13:42.488014 28805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 742725f311d64b53b1eddd4cabaab3e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:42.494457 28994 tablet_bootstrap.cc:492] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11: Bootstrap starting.
I20260806 01:13:42.498062 28993 tablet_bootstrap.cc:492] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b: Bootstrap starting.
I20260806 01:13:42.498178 28994 tablet_bootstrap.cc:654] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:42.500387 28993 tablet_bootstrap.cc:654] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:42.501176 28994 tablet_bootstrap.cc:492] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11: No bootstrap required, opened a new log
I20260806 01:13:42.501392 28994 ts_tablet_manager.cc:1397] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.001s
I20260806 01:13:42.502538 28994 raft_consensus.cc:348] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.503104 28994 raft_consensus.cc:374] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:42.503214 28994 raft_consensus.cc:729] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c09dd495f2554c9baeacc7c0a5c89b11, State: Initialized, Role: FOLLOWER
I20260806 01:13:42.503733 28997 tablet_bootstrap.cc:492] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096: Bootstrap starting.
I20260806 01:13:42.503635 28994 consensus_queue.cc:260] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.504200 28970 tablet_replica.cc:439] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } }
I20260806 01:13:42.504532 28970 tablet_replica.cc:442] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:42.505254 28994 ts_tablet_manager.cc:1428] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 01:13:42.505954 28997 tablet_bootstrap.cc:654] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:42.509069 28993 tablet_bootstrap.cc:492] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b: No bootstrap required, opened a new log
I20260806 01:13:42.509312 28993 ts_tablet_manager.cc:1397] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.004s
I20260806 01:13:42.510608 28993 raft_consensus.cc:348] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.510999 28993 raft_consensus.cc:374] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:42.511129 28993 raft_consensus.cc:729] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df60cc1933144209fe334035c7e262b, State: Initialized, Role: FOLLOWER
I20260806 01:13:42.511579 28993 consensus_queue.cc:260] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.512387 28993 ts_tablet_manager.cc:1428] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 01:13:42.512284 28960 tablet_replica.cc:439] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } }
I20260806 01:13:42.512807 28960 tablet_replica.cc:442] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:42.514652 28997 tablet_bootstrap.cc:492] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096: No bootstrap required, opened a new log
I20260806 01:13:42.514931 28997 ts_tablet_manager.cc:1397] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.003s
I20260806 01:13:42.515774 28997 raft_consensus.cc:348] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.516088 28997 raft_consensus.cc:374] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:42.516181 28997 raft_consensus.cc:729] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c43b5bb8e5e4843b89b60dcc0505096, State: Initialized, Role: FOLLOWER
I20260806 01:13:42.516474 28997 consensus_queue.cc:260] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.517132 28997 ts_tablet_manager.cc:1428] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 01:13:42.516985 28974 tablet_replica.cc:439] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } }
I20260806 01:13:42.517488 28974 tablet_replica.cc:442] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:42.618083 28970 raft_consensus.cc:482] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:13:42.618465 28970 raft_consensus.cc:504] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.619244 28970 leader_election.cc:288] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c43b5bb8e5e4843b89b60dcc0505096 (127.25.236.130:38945), 5df60cc1933144209fe334035c7e262b (127.25.236.131:44933)
I20260806 01:13:42.620054 28635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "742725f311d64b53b1eddd4cabaab3e0" candidate_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" is_pre_election: true
I20260806 01:13:42.620474 28635 raft_consensus.cc:2454] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c09dd495f2554c9baeacc7c0a5c89b11 in term 0.
I20260806 01:13:42.620671 28843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "742725f311d64b53b1eddd4cabaab3e0" candidate_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df60cc1933144209fe334035c7e262b" is_pre_election: true
I20260806 01:13:42.620988 28330 leader_election.cc:302] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c43b5bb8e5e4843b89b60dcc0505096, c09dd495f2554c9baeacc7c0a5c89b11; no voters:
I20260806 01:13:42.621023 28843 raft_consensus.cc:2454] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c09dd495f2554c9baeacc7c0a5c89b11 in term 0.
I20260806 01:13:42.621333 28970 raft_consensus.cc:2781] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:13:42.621456 28970 raft_consensus.cc:482] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:13:42.621585 28970 raft_consensus.cc:3037] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:42.625188 28970 raft_consensus.cc:504] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.625893 28970 leader_election.cc:288] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [CANDIDATE]: Term 1 election: Requested vote from peers 7c43b5bb8e5e4843b89b60dcc0505096 (127.25.236.130:38945), 5df60cc1933144209fe334035c7e262b (127.25.236.131:44933)
I20260806 01:13:42.626413 28635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "742725f311d64b53b1eddd4cabaab3e0" candidate_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c43b5bb8e5e4843b89b60dcc0505096"
I20260806 01:13:42.626542 28843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "742725f311d64b53b1eddd4cabaab3e0" candidate_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df60cc1933144209fe334035c7e262b"
I20260806 01:13:42.626659 28635 raft_consensus.cc:3037] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:42.626799 28843 raft_consensus.cc:3037] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:42.629027 28843 raft_consensus.cc:2454] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c09dd495f2554c9baeacc7c0a5c89b11 in term 1.
I20260806 01:13:42.629562 28330 leader_election.cc:302] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5df60cc1933144209fe334035c7e262b, c09dd495f2554c9baeacc7c0a5c89b11; no voters:
I20260806 01:13:42.629844 28970 raft_consensus.cc:2781] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:42.629940 28635 raft_consensus.cc:2454] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c09dd495f2554c9baeacc7c0a5c89b11 in term 1.
I20260806 01:13:42.630050 28970 raft_consensus.cc:686] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [term 1 LEADER]: Becoming Leader. State: Replica: c09dd495f2554c9baeacc7c0a5c89b11, State: Running, Role: LEADER
I20260806 01:13:42.630445 28970 consensus_queue.cc:237] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } }
I20260806 01:13:42.632678 29003 tablet_replica.cc:439] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11: TxnStatusTablet state changed. Reason: New leader c09dd495f2554c9baeacc7c0a5c89b11. Latest consensus state: current_term: 1 leader_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } }
I20260806 01:13:42.633041 29003 tablet_replica.cc:442] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:13:42.634784 29025 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 01:13:42.634642 28215 catalog_manager.cc:5495] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 reported cstate change: term changed from 0 to 1, leader changed from <none> to c09dd495f2554c9baeacc7c0a5c89b11 (127.25.236.129). New cstate: current_term: 1 leader_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } health_report { overall_health: UNKNOWN } } }
I20260806 01:13:42.635115 29025 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 01:13:42.675452 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.665630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:35425 (local address 127.0.0.1:47484)
0806 01:13:42.665926 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:42.665950 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:42.666108 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:42.667077 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:42.667082 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:42.667095 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:13:42.667446 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:42.667461 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.668995 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.669003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:42.671087 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:42.671120 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.672281 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.672288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:42.672608 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.672709 (+ 101us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:13:42.674202 (+ 1493us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:13:42.674328 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 01:13:42.674457 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 01:13:42.674657 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 01:13:42.675490 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.665625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47484 (local address 127.25.236.129:35425)
0806 01:13:42.666613 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:42.666618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:42.666678 (+ 60us) server_negotiation.cc:407] Connection header received
0806 01:13:42.666780 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:42.666784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:42.666887 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:42.667057 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:42.667598 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.668860 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.671424 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.672051 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.672255 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.673318 (+ 1063us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:42.674034 (+ 716us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:42.674375 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:42.674473 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 01:13:42.674565 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":545,"threads_started":1}
I20260806 01:13:42.682826 28843 raft_consensus.cc:1261] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b [term 1 FOLLOWER]: Refusing update from remote peer c09dd495f2554c9baeacc7c0a5c89b11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:42.682787 28635 raft_consensus.cc:1261] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096 [term 1 FOLLOWER]: Refusing update from remote peer c09dd495f2554c9baeacc7c0a5c89b11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:42.683851 29003 consensus_queue.cc:1035] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:42.684302 28970 consensus_queue.cc:1035] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:42.690644 28974 tablet_replica.cc:439] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096: TxnStatusTablet state changed. Reason: New leader c09dd495f2554c9baeacc7c0a5c89b11. Latest consensus state: current_term: 1 leader_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } }
I20260806 01:13:42.691051 28974 tablet_replica.cc:442] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:42.693662 29032 mvcc.cc:205] Tried to move back new op lower bound from 7315369257698324480 to 7315369257496899584. Current Snapshot: MvccSnapshot[applied={T|T < 7315369257698324480}]
I20260806 01:13:42.693974 28960 tablet_replica.cc:439] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b: TxnStatusTablet state changed. Reason: New leader c09dd495f2554c9baeacc7c0a5c89b11. Latest consensus state: current_term: 1 leader_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } }
I20260806 01:13:42.694438 28960 tablet_replica.cc:442] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:42.699725 28974 tablet_replica.cc:439] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } }
I20260806 01:13:42.700027 28974 tablet_replica.cc:442] T 742725f311d64b53b1eddd4cabaab3e0 P 7c43b5bb8e5e4843b89b60dcc0505096: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:42.700567 28960 tablet_replica.cc:439] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } }
I20260806 01:13:42.700850 28960 tablet_replica.cc:442] T 742725f311d64b53b1eddd4cabaab3e0 P 5df60cc1933144209fe334035c7e262b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:42.704106 29033 mvcc.cc:205] Tried to move back new op lower bound from 7315369257698324480 to 7315369257496899584. Current Snapshot: MvccSnapshot[applied={T|T < 7315369257698324480}]
I20260806 01:13:42.706804 28970 tablet_replica.cc:439] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } }
I20260806 01:13:42.707501 29003 tablet_replica.cc:439] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09dd495f2554c9baeacc7c0a5c89b11" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35425 } } peers { permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 } } peers { permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 } } }
I20260806 01:13:42.707981 28970 tablet_replica.cc:442] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:13:42.708127 29003 tablet_replica.cc:442] T 742725f311d64b53b1eddd4cabaab3e0 P c09dd495f2554c9baeacc7c0a5c89b11: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:13:42.724951 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.716954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47492 (local address 127.25.236.129:35425)
0806 01:13:42.717130 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:42.717135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:42.718263 (+ 1128us) server_negotiation.cc:407] Connection header received
0806 01:13:42.718725 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:42.718731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:42.718835 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:42.718952 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:42.719662 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.720821 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.722578 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.723316 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.723578 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.724136 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:42.724276 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:42.724388 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:42.724531 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 01:13:42.724623 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 01:13:42.748410 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.739418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:39063 (local address 127.25.236.129:35425)
0806 01:13:42.739723 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:42.739732 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:42.740913 (+ 1181us) server_negotiation.cc:407] Connection header received
0806 01:13:42.741167 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:42.741171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:42.741296 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:42.741381 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:42.742348 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.743225 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.745808 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.746518 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.746619 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.747314 (+ 695us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:42.747422 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:42.747497 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:42.748073 (+ 576us) server_negotiation.cc:299] Negotiation successful
0806 01:13:42.748156 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":88}
I20260806 01:13:42.748561 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.739670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:35425 (local address 127.25.236.129:39063)
0806 01:13:42.740737 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:42.740761 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:42.740957 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:42.741423 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:42.741434 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:42.741447 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:13:42.742007 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:42.742021 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.743451 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.743458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:42.745631 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:42.745649 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.746752 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.746760 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:42.746993 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.747108 (+ 115us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:13:42.747543 (+ 435us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:13:42.747558 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 01:13:42.747660 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 01:13:42.747822 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":20,"thread_start_us":586,"threads_started":1}
I20260806 01:13:42.760648 28635 raft_consensus.cc:1261] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096 [term 1 FOLLOWER]: Refusing update from remote peer c09dd495f2554c9baeacc7c0a5c89b11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:42.760679 28840 raft_consensus.cc:1261] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b [term 1 FOLLOWER]: Refusing update from remote peer c09dd495f2554c9baeacc7c0a5c89b11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:42.761297 29003 consensus_queue.cc:1035] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5df60cc1933144209fe334035c7e262b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:42.761683 28970 consensus_queue.cc:1035] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c43b5bb8e5e4843b89b60dcc0505096" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:42.853405 28215 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:13:42.853790 28215 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:13:42.857722 28215 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8eabf5f59644edd9ea73c5ad2a25858: Sending DeleteTablet for 3 replicas of tablet 39822454beee4b6c9b8f869a97c8556b
I20260806 01:13:42.859153 28805 tablet_service.cc:1514] Processing DeleteTablet for tablet 39822454beee4b6c9b8f869a97c8556b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:42 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:50924
I20260806 01:13:42.859287 28602 tablet_service.cc:1514] Processing DeleteTablet for tablet 39822454beee4b6c9b8f869a97c8556b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:42 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:49804
I20260806 01:13:42.859387 28402 tablet_service.cc:1514] Processing DeleteTablet for tablet 39822454beee4b6c9b8f869a97c8556b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:42 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:47474
I20260806 01:13:42.872062 29057 tablet_replica.cc:330] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096: stopping tablet replica
I20260806 01:13:42.872663 29057 raft_consensus.cc:2229] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:13:42.873129 29057 raft_consensus.cc:2258] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:42.874079 29058 tablet_replica.cc:330] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11: stopping tablet replica
I20260806 01:13:42.874414 29058 raft_consensus.cc:2229] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:13:42.875219 29058 raft_consensus.cc:2258] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:42.876210 29057 ts_tablet_manager.cc:1905] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:42.879302 29058 ts_tablet_manager.cc:1905] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:42.885229 29057 ts_tablet_manager.cc:1918] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:13:42.885445 29057 log.cc:1196] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/39822454beee4b6c9b8f869a97c8556b
I20260806 01:13:42.885985 29057 ts_tablet_manager.cc:1939] T 39822454beee4b6c9b8f869a97c8556b P 7c43b5bb8e5e4843b89b60dcc0505096: Deleting consensus metadata
I20260806 01:13:42.886886 29056 tablet_replica.cc:330] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b: stopping tablet replica
I20260806 01:13:42.887135 29056 raft_consensus.cc:2229] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:13:42.887403 29056 raft_consensus.cc:2258] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:42.887362 28198 catalog_manager.cc:4841] TS 7c43b5bb8e5e4843b89b60dcc0505096 (127.25.236.130:38945): tablet 39822454beee4b6c9b8f869a97c8556b (table test-table [id=e75fa24d9bec41fea5bf108cdde983df]) successfully deleted
I20260806 01:13:42.889492 29058 ts_tablet_manager.cc:1918] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:13:42.889871 29058 log.cc:1196] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/39822454beee4b6c9b8f869a97c8556b
I20260806 01:13:42.889900 29056 ts_tablet_manager.cc:1905] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:42.890760 29058 ts_tablet_manager.cc:1939] T 39822454beee4b6c9b8f869a97c8556b P c09dd495f2554c9baeacc7c0a5c89b11: Deleting consensus metadata
I20260806 01:13:42.892884 28192 catalog_manager.cc:4841] TS c09dd495f2554c9baeacc7c0a5c89b11 (127.25.236.129:35425): tablet 39822454beee4b6c9b8f869a97c8556b (table test-table [id=e75fa24d9bec41fea5bf108cdde983df]) successfully deleted
I20260806 01:13:42.898952 29056 ts_tablet_manager.cc:1918] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:13:42.899116 29056 log.cc:1196] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/39822454beee4b6c9b8f869a97c8556b
I20260806 01:13:42.899724 29056 ts_tablet_manager.cc:1939] T 39822454beee4b6c9b8f869a97c8556b P 5df60cc1933144209fe334035c7e262b: Deleting consensus metadata
I20260806 01:13:42.901027 28198 catalog_manager.cc:4841] TS 5df60cc1933144209fe334035c7e262b (127.25.236.131:44933): tablet 39822454beee4b6c9b8f869a97c8556b (table test-table [id=e75fa24d9bec41fea5bf108cdde983df]) successfully deleted
Aug 06 01:13:42 dist-test-slave-7nm7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978821, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:13:42.910734 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.884235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47508 (local address 127.25.236.129:35425)
0806 01:13:42.884478 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:42.884494 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:42.888530 (+ 4036us) server_negotiation.cc:407] Connection header received
0806 01:13:42.888774 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:42.888780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:42.888863 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:42.888962 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:42.892465 (+ 3503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.894131 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.896551 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.897418 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.897683 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.903178 (+ 5495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:42.903195 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:42.903212 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:42.903252 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:42.907092 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:42.908038 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:42.908046 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:42.908048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:42.908198 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:42.908662 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:42.908666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:42.908668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:42.909186 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:42.909666 (+ 480us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:42.909883 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 01:13:42.910130 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260806 01:13:42.911934 28496 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47508
I20260806 01:13:42.932955 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.916175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40398 (local address 127.25.236.190:40135)
0806 01:13:42.916598 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:42.916608 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:42.916761 (+ 153us) server_negotiation.cc:407] Connection header received
0806 01:13:42.916996 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:42.917000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:42.917066 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:42.917164 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:42.919272 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.920023 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.921866 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.922403 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.922510 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.925956 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:42.925962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:42.925965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:42.926035 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:42.929264 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:42.930335 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:42.930340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:42.930342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:42.930464 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:42.931033 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:42.931037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:42.931039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:42.931368 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:42.931557 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:42.932325 (+ 768us) server_negotiation.cc:299] Negotiation successful
0806 01:13:42.932632 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250}
W20260806 01:13:42.933660 28215 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40398
I20260806 01:13:42.937017 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 28307
I20260806 01:13:42.953941 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.938162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47516 (local address 127.25.236.129:35425)
0806 01:13:42.938379 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:42.938385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:42.938722 (+ 337us) server_negotiation.cc:407] Connection header received
0806 01:13:42.938864 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:42.938872 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:42.938947 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:42.939070 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:42.940323 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.941572 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.943017 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.943774 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.943959 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:42.946507 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:42.946514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:42.946519 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:42.946558 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:42.950467 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:42.951385 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:42.951393 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:42.951396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:42.951698 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:42.952290 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:42.952299 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:42.952309 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:42.952750 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:42.953022 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:42.953333 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 01:13:42.953590 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260806 01:13:42.954704 28496 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47516
W20260806 01:13:42.955650 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 28307: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:13:42.955864 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 28307
I20260806 01:13:42.991765 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 28507
Aug 06 01:13:43 dist-test-slave-7nm7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978821, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:13:43.009783 29067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:42.992615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49818 (local address 127.25.236.130:38945)
0806 01:13:42.993319 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:42.993327 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:42.993352 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:13:42.993408 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:42.993430 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:42.993645 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:42.993765 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:42.995683 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.996738 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.998226 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:42.998738 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:42.998852 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:43.003482 (+ 4630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:43.003497 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:43.003514 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:43.003550 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:43.006896 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:43.007627 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:43.007635 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:43.007637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:43.007901 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:43.008354 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:43.008358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:43.008360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:43.008727 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:43.008958 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:43.009254 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 01:13:43.009431 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":406,"threads_started":1}
W20260806 01:13:43.010973 28696 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49818
W20260806 01:13:43.011888 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 28507: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:13:43.012081 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 28507
I20260806 01:13:43.039211 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 28707
Aug 06 01:13:43 dist-test-slave-7nm7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978821, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:13:43.057940 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:43.040022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50940 (local address 127.25.236.131:44933)
0806 01:13:43.040600 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:43.040605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:43.040629 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:13:43.040687 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:43.040692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:43.040762 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:43.040856 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:43.042392 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:43.043553 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:43.044869 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:43.045293 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:43.045393 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:43.050559 (+ 5166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:43.050585 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:43.050613 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:43.050663 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:43.054183 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:43.055468 (+ 1285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:43.055476 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:43.055478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:43.055631 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:43.056204 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:43.056209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:43.056211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:43.056634 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:43.056874 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:43.057223 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 01:13:43.057521 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":368,"threads_started":1}
W20260806 01:13:43.059145 28902 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50940
W20260806 01:13:43.060048 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 28707: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:13:43.060232 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 28707
I20260806 01:13:43.090417 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 28167
I20260806 01:13:43.091621 28268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:43.212440 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 28167
2026-08-06T01:13:43Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (6005 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:13:43 dist-test-slave-7nm7 krb5kdc[29076](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:13:43 dist-test-slave-7nm7 krb5kdc[29076](info): set up 2 sockets
Aug 06 01:13:43 dist-test-slave-7nm7 krb5kdc[29076](info): commencing operation
krb5kdc: starting...
W20260806 01:13:44.554354 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:13:44 dist-test-slave-7nm7 krb5kdc[29076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:13:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:13:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:13:44.795444 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:37343
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:37343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:45.158905 29094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:45.159245 29094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:45.159401 29094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:45.167208 29094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:45.167377 29094 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 01:13:45.167479 29094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:45.167603 29094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:45.167696 29094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:45.167815 29094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:45.178469 29094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:37343
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:37343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:45.179783 29094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:45.181280 29094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:45.193715 29106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:45.193991 29102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:45.194169 29101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:45.194585 29094 server_base.cc:1034] running on GCE node
I20260806 01:13:45.195948 29094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:45.197741 29094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:45.199054 29094 hybrid_clock.cc:648] HybridClock initialized: now 1785978825198966 us; error 52 us; skew 500 ppm
I20260806 01:13:45.199450 29094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:45 dist-test-slave-7nm7 krb5kdc[29076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978825, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:45.208288 29094 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:13:45.210108 29094 webserver.cc:460] Webserver started at http://127.25.236.190:32969/ using document root <none> and password file <none>
I20260806 01:13:45.210932 29094 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:45.211097 29094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:45.211467 29094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:45.215250 29094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "7fb91a3ee2f74327bd283642b185fe3d"
format_stamp: "Formatted at 2026-08-06 01:13:45 on dist-test-slave-7nm7"
server_key: "dae77c00ec184f476a72ebb28c0b5cb7"
server_key_iv: "d408166d3626d0beb250b8624181d224"
server_key_version: "encryptionkey@0"
I20260806 01:13:45.216660 29094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "7fb91a3ee2f74327bd283642b185fe3d"
format_stamp: "Formatted at 2026-08-06 01:13:45 on dist-test-slave-7nm7"
server_key: "dae77c00ec184f476a72ebb28c0b5cb7"
server_key_iv: "d408166d3626d0beb250b8624181d224"
server_key_version: "encryptionkey@0"
I20260806 01:13:45.223627 29094 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260806 01:13:45.227721 29117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:45.229382 29094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260806 01:13:45.229580 29094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "7fb91a3ee2f74327bd283642b185fe3d"
format_stamp: "Formatted at 2026-08-06 01:13:45 on dist-test-slave-7nm7"
server_key: "dae77c00ec184f476a72ebb28c0b5cb7"
server_key_iv: "d408166d3626d0beb250b8624181d224"
server_key_version: "encryptionkey@0"
I20260806 01:13:45.229827 29094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:45.244689 29094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:45.249927 29094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:45.250324 29094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:45.277521 29094 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:37343
I20260806 01:13:45.277532 29210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:37343 every 8 connection(s)
I20260806 01:13:45.279145 29094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:13:45.281271 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 29094
I20260806 01:13:45.281541 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:13:45.282769 26546 external_mini_cluster.cc:1428] Setting key f0cd562ac632656d4058c198a621769d
I20260806 01:13:45.285722 29211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:13:45 dist-test-slave-7nm7 krb5kdc[29076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:45.303612 29211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d: Bootstrap starting.
I20260806 01:13:45.307891 29211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:45.309290 29211 log.cc:824] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:45.311079 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:45.285199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41022 (local address 127.25.236.190:37343)
0806 01:13:45.286115 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:45.286128 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:45.286172 (+ 44us) server_negotiation.cc:407] Connection header received
0806 01:13:45.287994 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:45.288016 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:45.288424 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:45.288782 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:45.290838 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:45.292273 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:45.294461 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:45.295589 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:45.295733 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:45.301912 (+ 6179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:45.301927 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:45.301951 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:45.302007 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:45.306287 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:45.307069 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:45.307078 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:45.307086 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:45.307303 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:45.307719 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:45.307724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:45.307727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:45.308528 (+ 801us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:45.308808 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:45.309290 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 01:13:45.309738 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":437,"threads_started":1}
I20260806 01:13:45.312757 29211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d: No bootstrap required, opened a new log
I20260806 01:13:45.321556 29211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb91a3ee2f74327bd283642b185fe3d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37343 } }
I20260806 01:13:45.321979 29211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:45.322085 29211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb91a3ee2f74327bd283642b185fe3d, State: Initialized, Role: FOLLOWER
I20260806 01:13:45.322764 29211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb91a3ee2f74327bd283642b185fe3d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37343 } }
I20260806 01:13:45.323015 29211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:13:45.323153 29211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:13:45.323618 29211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:45.326637 29211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb91a3ee2f74327bd283642b185fe3d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37343 } }
I20260806 01:13:45.327193 29211 leader_election.cc:302] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fb91a3ee2f74327bd283642b185fe3d; no voters:
I20260806 01:13:45.327782 29211 leader_election.cc:288] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:13:45.327934 29217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:45.328778 29217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [term 1 LEADER]: Becoming Leader. State: Replica: 7fb91a3ee2f74327bd283642b185fe3d, State: Running, Role: LEADER
I20260806 01:13:45.329494 29217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb91a3ee2f74327bd283642b185fe3d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37343 } }
I20260806 01:13:45.330031 29211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:45.333628 29218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fb91a3ee2f74327bd283642b185fe3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb91a3ee2f74327bd283642b185fe3d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37343 } } }
I20260806 01:13:45.334220 29218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:45.335073 29219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fb91a3ee2f74327bd283642b185fe3d. Latest consensus state: current_term: 1 leader_uuid: "7fb91a3ee2f74327bd283642b185fe3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb91a3ee2f74327bd283642b185fe3d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37343 } } }
I20260806 01:13:45.335817 29219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:45.339445 29224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:13:45.345961 29224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:13:45.356555 29224 catalog_manager.cc:1347] Generated new cluster ID: 50b7c2b1197542efb7bad43e0119dc2c
I20260806 01:13:45.356762 29224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:13:45.379422 29224 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:13:45.381055 29224 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:13:45.391147 29224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d: Generated new TSK 0
I20260806 01:13:45.391914 29224 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:13:45.447691 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:37343
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:45.815804 29247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:45.816138 29247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:45.816285 29247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:45.824455 29247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:45.824605 29247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:45.824795 29247 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 01:13:45.824865 29247 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:13:45.824976 29247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:13:45.836498 29247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.236.190:37343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:45.837896 29247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:45.839430 29247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:45.855861 29256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:45.855988 29257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:45.856087 29259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:45.857465 29247 server_base.cc:1034] running on GCE node
I20260806 01:13:45.858660 29247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:45.859787 29247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:45.861016 29247 hybrid_clock.cc:648] HybridClock initialized: now 1785978825860936 us; error 63 us; skew 500 ppm
I20260806 01:13:45.861349 29247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:45 dist-test-slave-7nm7 krb5kdc[29076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978825, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:45.865216 29247 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:13:45.867029 29247 webserver.cc:460] Webserver started at http://127.25.236.129:40705/ using document root <none> and password file <none>
I20260806 01:13:45.867750 29247 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:45.867888 29247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:45.868237 29247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:45.871299 29247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "c409779bcd184007bb77e07b0881fb20"
format_stamp: "Formatted at 2026-08-06 01:13:45 on dist-test-slave-7nm7"
server_key: "81a95902f25bb9c448485fea946844c4"
server_key_iv: "2c71a8fde53523af176d8d64bc16dbc2"
server_key_version: "encryptionkey@0"
I20260806 01:13:45.872095 29247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "c409779bcd184007bb77e07b0881fb20"
format_stamp: "Formatted at 2026-08-06 01:13:45 on dist-test-slave-7nm7"
server_key: "81a95902f25bb9c448485fea946844c4"
server_key_iv: "2c71a8fde53523af176d8d64bc16dbc2"
server_key_version: "encryptionkey@0"
I20260806 01:13:45.877736 29247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:13:45.881737 29267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:45.883314 29247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:13:45.883512 29247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "c409779bcd184007bb77e07b0881fb20"
format_stamp: "Formatted at 2026-08-06 01:13:45 on dist-test-slave-7nm7"
server_key: "81a95902f25bb9c448485fea946844c4"
server_key_iv: "2c71a8fde53523af176d8d64bc16dbc2"
server_key_version: "encryptionkey@0"
I20260806 01:13:45.883736 29247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:45.905319 29247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:45.910094 29247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:45.910458 29247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:45.913450 29247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:45.913586 29247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:45.913728 29247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:45.913833 29247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:13:45 dist-test-slave-7nm7 krb5kdc[29076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978825, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:45.942497 29289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:45.914172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:49181 (local address 127.25.236.190:37343)
0806 01:13:45.914973 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:45.914980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:45.916188 (+ 1208us) server_negotiation.cc:407] Connection header received
0806 01:13:45.918441 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:45.918448 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:45.918542 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:45.918677 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:45.921555 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:45.922733 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:45.925521 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:45.926163 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:45.926387 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:45.930939 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:45.930946 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:45.930950 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:45.931044 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:45.936313 (+ 5269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:45.937786 (+ 1473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:45.937795 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:45.937801 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:45.937973 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:45.939578 (+ 1605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:45.939584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:45.939588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:45.939961 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:45.940258 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:45.941549 (+ 1291us) server_negotiation.cc:299] Negotiation successful
0806 01:13:45.941913 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":550,"threads_started":1}
I20260806 01:13:45.943495 29290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:45.915002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:37343 (local address 127.25.236.129:49181)
0806 01:13:45.915974 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:45.916031 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:45.918087 (+ 2056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:45.918916 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:45.918925 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:45.919787 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:45.921226 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:45.921261 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:45.922940 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:45.922947 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:45.925247 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:45.925272 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:45.926555 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:45.926565 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:45.926819 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:45.927640 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:45.927674 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:45.930630 (+ 2956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:45.936555 (+ 5925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:45.936573 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:45.936595 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:45.937576 (+ 981us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:45.938382 (+ 806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:45.938391 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:45.938399 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:45.939076 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:45.940258 (+ 1182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:45.940269 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:45.940892 (+ 623us) client_negotiation.cc:769] Sending connection context
0806 01:13:45.941385 (+ 493us) client_negotiation.cc:240] Negotiation successful
0806 01:13:45.941938 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":381,"threads_started":1}
I20260806 01:13:45.967679 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:45.949863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:37343 (local address 127.25.236.129:33047)
0806 01:13:45.950931 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:45.950957 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:45.951158 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:45.951578 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:45.951584 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:45.952483 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:45.952948 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:45.952967 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:45.954937 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:45.954945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:45.956435 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:45.956450 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:45.957462 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:45.957470 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:45.957652 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:45.958382 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:45.958406 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:45.959811 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:45.963881 (+ 4070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:45.963891 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:45.963903 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:45.964391 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:45.964900 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:45.964906 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:45.964910 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:45.965104 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:45.965872 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:45.965880 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:45.966168 (+ 288us) client_negotiation.cc:769] Sending connection context
0806 01:13:45.966379 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 01:13:45.966899 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"thread_start_us":649,"threads_started":1}
I20260806 01:13:45.967708 29289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:45.949836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:33047 (local address 127.25.236.190:37343)
0806 01:13:45.950061 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:45.950070 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:45.951075 (+ 1005us) server_negotiation.cc:407] Connection header received
0806 01:13:45.951295 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:45.951299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:45.951369 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:45.951510 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:45.953392 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:45.954752 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:45.956705 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:45.957223 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:45.957377 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:45.960028 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:45.960033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:45.960036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:45.960099 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:45.963715 (+ 3616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:45.964562 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:45.964568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:45.964570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:45.964704 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:45.965305 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:45.965311 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:45.965317 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:45.965679 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:45.965891 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:45.966900 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0806 01:13:45.967214 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 01:13:45.977878 29247 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:35167
I20260806 01:13:45.977919 29462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:35167 every 8 connection(s)
I20260806 01:13:45.979202 29247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:13:45.981464 29463 heartbeater.cc:347] Connected to a master server at 127.25.236.190:37343
I20260806 01:13:45.981734 29463 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:45.982710 29463 heartbeater.cc:511] Master 127.25.236.190:37343 requested a full tablet report, sending...
I20260806 01:13:45.984889 29147 ts_manager.cc:194] Registered new tserver with Master: c409779bcd184007bb77e07b0881fb20 (127.25.236.129:35167)
I20260806 01:13:45.985033 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 29247
I20260806 01:13:45.985291 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:13:45.985991 26546 external_mini_cluster.cc:1428] Setting key ab837328d87193ee626275c0be426eee
I20260806 01:13:45.986847 29147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:49181
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:13:46.042351 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:37343
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:46.401278 29468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:46.401626 29468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:46.401804 29468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:46.409642 29468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:46.409816 29468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:46.409958 29468 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 01:13:46.410044 29468 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:13:46.410175 29468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:13:46.421015 29468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.236.190:37343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:46.422416 29468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:46.423725 29468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:46.437394 29477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:46.437458 29481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:46.437402 29478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:46.438328 29468 server_base.cc:1034] running on GCE node
I20260806 01:13:46.438997 29468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:46.440047 29468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:46.441335 29468 hybrid_clock.cc:648] HybridClock initialized: now 1785978826441259 us; error 59 us; skew 500 ppm
I20260806 01:13:46.441790 29468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:46 dist-test-slave-7nm7 krb5kdc[29076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978826, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:46.446125 29468 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:13:46.447585 29468 webserver.cc:460] Webserver started at http://127.25.236.130:34861/ using document root <none> and password file <none>
I20260806 01:13:46.448216 29468 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:46.448318 29468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:46.448576 29468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:46.451490 29468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "048d166145f0424cbd94bca72db483b4"
format_stamp: "Formatted at 2026-08-06 01:13:46 on dist-test-slave-7nm7"
server_key: "cfa0a0d824768ff2e093e2fcc3cc5266"
server_key_iv: "5b474c9035d1f433b5e3a6906c91303a"
server_key_version: "encryptionkey@0"
I20260806 01:13:46.452194 29468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "048d166145f0424cbd94bca72db483b4"
format_stamp: "Formatted at 2026-08-06 01:13:46 on dist-test-slave-7nm7"
server_key: "cfa0a0d824768ff2e093e2fcc3cc5266"
server_key_iv: "5b474c9035d1f433b5e3a6906c91303a"
server_key_version: "encryptionkey@0"
I20260806 01:13:46.457216 29468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:13:46.460752 29491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:46.462605 29468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:13:46.462810 29468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "048d166145f0424cbd94bca72db483b4"
format_stamp: "Formatted at 2026-08-06 01:13:46 on dist-test-slave-7nm7"
server_key: "cfa0a0d824768ff2e093e2fcc3cc5266"
server_key_iv: "5b474c9035d1f433b5e3a6906c91303a"
server_key_version: "encryptionkey@0"
I20260806 01:13:46.463028 29468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:46.492074 29468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:46.496418 29468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:46.496727 29468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:46.500353 29468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:46.500500 29468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:46.500651 29468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:46.500751 29468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:13:46 dist-test-slave-7nm7 krb5kdc[29076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978826, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:46.526976 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:46.500603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:54281 (local address 127.25.236.190:37343)
0806 01:13:46.501200 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:46.501209 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:46.502602 (+ 1393us) server_negotiation.cc:407] Connection header received
0806 01:13:46.504786 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:46.504795 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:46.504911 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:46.505055 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:46.507697 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:46.508893 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:46.512283 (+ 3390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:46.512874 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:46.513029 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:46.519501 (+ 6472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:46.519510 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:46.519516 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:46.519559 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:46.523015 (+ 3456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:46.524060 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:46.524066 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:46.524072 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:46.524245 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:46.524887 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:46.524892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:46.524895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:46.525183 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:46.525355 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:46.526349 (+ 994us) server_negotiation.cc:299] Negotiation successful
0806 01:13:46.526585 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":290,"threads_started":1}
I20260806 01:13:46.528870 29505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:46.501194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:37343 (local address 127.25.236.130:54281)
0806 01:13:46.502403 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:46.502459 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:46.504479 (+ 2020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:46.505356 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:46.505368 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:46.506290 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:46.507411 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:46.507448 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:46.509152 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:46.509165 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:46.512058 (+ 2893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:46.512079 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:46.513198 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:46.513212 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:46.513600 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:46.514578 (+ 978us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:46.514619 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:46.519161 (+ 4542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:46.523172 (+ 4011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:46.523181 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:46.523200 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:46.523867 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:46.524370 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:46.524379 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:46.524385 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:46.524753 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:46.525325 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:46.525336 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:46.525834 (+ 498us) client_negotiation.cc:769] Sending connection context
0806 01:13:46.526212 (+ 378us) client_negotiation.cc:240] Negotiation successful
0806 01:13:46.526825 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"thread_start_us":424,"threads_started":1}
I20260806 01:13:46.551468 29468 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:36681
I20260806 01:13:46.551525 29666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:36681 every 8 connection(s)
I20260806 01:13:46.553112 29468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:13:46.553398 29626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:46.537140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:37343 (local address 127.25.236.130:56217)
0806 01:13:46.537829 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:46.537853 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:46.538017 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:46.538420 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:46.538424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:46.539330 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:46.539890 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:46.539903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:46.541089 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:46.541093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:46.543013 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:46.543030 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:46.544018 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:46.544026 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:46.544188 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:46.544935 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:46.544954 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:46.545786 (+ 832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:46.550407 (+ 4621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:46.550412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:46.550416 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:46.551060 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:46.551528 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:46.551532 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:46.551537 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:46.551706 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:46.552423 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:46.552427 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:46.552598 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 01:13:46.552716 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 01:13:46.552973 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":399,"threads_started":1}
I20260806 01:13:46.553433 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:46.537129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:56217 (local address 127.25.236.190:37343)
0806 01:13:46.537367 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:46.537376 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:46.537950 (+ 574us) server_negotiation.cc:407] Connection header received
0806 01:13:46.538157 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:46.538162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:46.538274 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:46.538388 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:46.540043 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:46.540921 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:46.543173 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:46.543818 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:46.543967 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:46.546076 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:46.546086 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:46.546094 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:46.546145 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:46.550269 (+ 4124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:46.551214 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:46.551223 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:46.551229 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:46.551392 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:46.551829 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:46.551834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:46.551837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:46.552291 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:46.552453 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:46.552842 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 01:13:46.553083 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:13:46.555354 29667 heartbeater.cc:347] Connected to a master server at 127.25.236.190:37343
I20260806 01:13:46.555605 29667 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:46.556267 29667 heartbeater.cc:511] Master 127.25.236.190:37343 requested a full tablet report, sending...
I20260806 01:13:46.557824 29147 ts_manager.cc:194] Registered new tserver with Master: 048d166145f0424cbd94bca72db483b4 (127.25.236.130:36681)
I20260806 01:13:46.558701 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 29468
I20260806 01:13:46.558887 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:13:46.558982 29147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:54281
I20260806 01:13:46.559484 26546 external_mini_cluster.cc:1428] Setting key e58a8af20e5ca5d8cab9c8d6e9e6784c
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:13:46.612861 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:37343
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:46.984721 29672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:46.985047 29672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:46.985169 29672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 01:13:46.990469 29463 heartbeater.cc:503] Master 127.25.236.190:37343 was elected leader, sending a full tablet report...
W20260806 01:13:46.993862 29672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:46.993988 29672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:46.994078 29672 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 01:13:46.994133 29672 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:13:46.994213 29672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:13:47.005362 29672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.236.190:37343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:47.006934 29672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:47.008374 29672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:47.023257 29681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:47.023373 29678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:47.023373 29679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:47.024310 29672 server_base.cc:1034] running on GCE node
I20260806 01:13:47.024987 29672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:47.025996 29672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:47.027249 29672 hybrid_clock.cc:648] HybridClock initialized: now 1785978827027211 us; error 73 us; skew 500 ppm
I20260806 01:13:47.027593 29672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:47 dist-test-slave-7nm7 krb5kdc[29076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978827, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:47.031034 29672 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:13:47.033116 29672 webserver.cc:460] Webserver started at http://127.25.236.131:34601/ using document root <none> and password file <none>
I20260806 01:13:47.033943 29672 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:47.034055 29672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:47.034412 29672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:47.037501 29672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "eec66fe008304da9a2e4a55e13a945af"
format_stamp: "Formatted at 2026-08-06 01:13:47 on dist-test-slave-7nm7"
server_key: "ba331a3c58f25e78ceb738a6a3445486"
server_key_iv: "5cd94c6c5b8047ee775c2aad473dc8fa"
server_key_version: "encryptionkey@0"
I20260806 01:13:47.038280 29672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "eec66fe008304da9a2e4a55e13a945af"
format_stamp: "Formatted at 2026-08-06 01:13:47 on dist-test-slave-7nm7"
server_key: "ba331a3c58f25e78ceb738a6a3445486"
server_key_iv: "5cd94c6c5b8047ee775c2aad473dc8fa"
server_key_version: "encryptionkey@0"
I20260806 01:13:47.045123 29672 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 01:13:47.050570 29692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:47.052145 29672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:13:47.052356 29672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "eec66fe008304da9a2e4a55e13a945af"
format_stamp: "Formatted at 2026-08-06 01:13:47 on dist-test-slave-7nm7"
server_key: "ba331a3c58f25e78ceb738a6a3445486"
server_key_iv: "5cd94c6c5b8047ee775c2aad473dc8fa"
server_key_version: "encryptionkey@0"
I20260806 01:13:47.052548 29672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:47.074388 29672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:47.078833 29672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:47.079164 29672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:47.081986 29672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:47.082283 29672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:47.082479 29672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:47.082584 29672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:13:47 dist-test-slave-7nm7 krb5kdc[29076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978827, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:47.108978 29704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.083008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:34385 (local address 127.25.236.190:37343)
0806 01:13:47.083798 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:47.083807 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:47.085055 (+ 1248us) server_negotiation.cc:407] Connection header received
0806 01:13:47.088291 (+ 3236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:47.088301 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:47.088401 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:47.088543 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:47.091091 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.092102 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.094700 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.095513 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.095664 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.100028 (+ 4364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:47.100035 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:47.100040 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:47.100086 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:47.104344 (+ 4258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:47.105410 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:47.105417 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:47.105423 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:47.105611 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:47.106314 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:47.106318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:47.106320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:47.106686 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:47.106862 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:47.108024 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0806 01:13:47.108410 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":522,"threads_started":1}
I20260806 01:13:47.111409 29705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.083690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:37343 (local address 127.25.236.131:34385)
0806 01:13:47.084842 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:47.084913 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:47.087848 (+ 2935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:47.088832 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:47.088849 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:47.089771 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:47.090854 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.090883 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.092356 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.092366 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.094475 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.094510 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.095837 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.095846 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.096148 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.096932 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:47.096976 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:47.099755 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:47.104561 (+ 4806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:47.104573 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:47.104608 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:47.105202 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:47.105736 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:47.105741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:47.105748 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:47.106120 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:47.106856 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:47.106872 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:47.107471 (+ 599us) client_negotiation.cc:769] Sending connection context
0806 01:13:47.107873 (+ 402us) client_negotiation.cc:240] Negotiation successful
0806 01:13:47.109011 (+ 1138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":532,"threads_started":1}
I20260806 01:13:47.134970 29704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.118075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:45811 (local address 127.25.236.190:37343)
0806 01:13:47.118299 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:47.118306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:47.118969 (+ 663us) server_negotiation.cc:407] Connection header received
0806 01:13:47.119173 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:47.119178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:47.119255 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:47.119363 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:47.120387 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.121099 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.123320 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.124154 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.124334 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.127105 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:47.127110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:47.127114 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:47.127145 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:47.131278 (+ 4133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:47.132220 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:47.132230 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:47.132236 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:47.132413 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:47.132936 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:47.132944 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:47.132950 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:47.133400 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:47.133573 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:47.134464 (+ 891us) server_negotiation.cc:299] Negotiation successful
0806 01:13:47.134659 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:13:47.135224 29820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.118097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:37343 (local address 127.25.236.131:45811)
0806 01:13:47.118863 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:47.118879 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:47.119042 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:47.119385 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:47.119390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:47.119945 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:47.120238 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.120247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.121268 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.121273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.122998 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.123022 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.124399 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.124406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.124642 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.125507 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:47.125533 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:47.126912 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:47.131477 (+ 4565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:47.131481 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:47.131484 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:47.132072 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:47.132565 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:47.132573 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:47.132579 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:47.132761 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:47.133557 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:47.133566 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:47.133885 (+ 319us) client_negotiation.cc:769] Sending connection context
0806 01:13:47.134221 (+ 336us) client_negotiation.cc:240] Negotiation successful
0806 01:13:47.134649 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":432,"threads_started":1}
I20260806 01:13:47.136775 29672 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:33229
I20260806 01:13:47.136861 29877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:33229 every 8 connection(s)
I20260806 01:13:47.138410 29672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:13:47.140779 29881 heartbeater.cc:347] Connected to a master server at 127.25.236.190:37343
I20260806 01:13:47.141028 29881 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:47.141700 29881 heartbeater.cc:511] Master 127.25.236.190:37343 requested a full tablet report, sending...
I20260806 01:13:47.143059 29147 ts_manager.cc:194] Registered new tserver with Master: eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229)
I20260806 01:13:47.143882 29147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:34385
I20260806 01:13:47.148651 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 29672
I20260806 01:13:47.148801 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:13:47.149230 26546 external_mini_cluster.cc:1428] Setting key 9019301672d87452e49d128c896e7eac
I20260806 01:13:47.152007 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:13:47 dist-test-slave-7nm7 krb5kdc[29076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:13:47 dist-test-slave-7nm7 krb5kdc[29076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:47.207119 29704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.191380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41026 (local address 127.25.236.190:37343)
0806 01:13:47.191627 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:47.191635 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:47.191916 (+ 281us) server_negotiation.cc:407] Connection header received
0806 01:13:47.192150 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:47.192154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:47.192232 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:47.192349 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:47.193748 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.194557 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.195778 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.196329 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.196491 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.201112 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:47.201123 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:47.201127 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:47.201171 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:47.204034 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:47.204794 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:47.204798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:47.204801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:47.204939 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:47.205408 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:47.205411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:47.205413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:47.205709 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:47.205918 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:47.206473 (+ 555us) server_negotiation.cc:299] Negotiation successful
0806 01:13:47.206843 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:13:47.211683 29147 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:13:47.219595 29147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:13:47.245666 29907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.237775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47712 (local address 127.25.236.130:36681)
0806 01:13:47.238842 (+ 1067us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:47.238850 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:47.238876 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:13:47.238971 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:47.238981 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:47.239219 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:47.239516 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:47.240142 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.241281 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.243638 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.244498 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.244687 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.244881 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:47.245149 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 01:13:47.245324 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":1444,"threads_started":1}
I20260806 01:13:47.246213 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.237290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:36681 (local address 127.0.0.1:47712)
0806 01:13:47.238078 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:47.238098 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:47.238293 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:47.239501 (+ 1208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:47.239505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:47.239531 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:47.239979 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.239989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.241432 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.241438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.243480 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.243492 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.244698 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.244705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.244916 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.244961 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 01:13:47.245096 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 01:13:47.245324 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":43,"thread_start_us":664,"threads_started":1}
I20260806 01:13:47.246835 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.234665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54026 (local address 127.25.236.129:35167)
0806 01:13:47.237486 (+ 2821us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:47.237495 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:47.237523 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:13:47.237618 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:47.237625 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:47.237841 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:47.238044 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:47.239000 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.240608 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.243741 (+ 3133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.245281 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.245449 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.245628 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:47.246390 (+ 762us) server_negotiation.cc:299] Negotiation successful
0806 01:13:47.246540 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2672,"thread_start_us":2536,"threads_started":1}
I20260806 01:13:47.247033 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.233881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:33229 (local address 127.0.0.1:57532)
0806 01:13:47.234738 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:47.234789 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:47.234976 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:47.237361 (+ 2385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:47.237369 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:47.237400 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:47.237895 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.237913 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.240762 (+ 2849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.240768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.243823 (+ 3055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.243835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.245650 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.245658 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.245852 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.245891 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 01:13:47.245955 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 01:13:47.246078 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":208,"thread_start_us":440,"threads_started":1}
I20260806 01:13:47.247139 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.234494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57532 (local address 127.25.236.131:33229)
0806 01:13:47.236662 (+ 2168us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:47.236670 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:47.236705 (+ 35us) server_negotiation.cc:407] Connection header received
0806 01:13:47.236818 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:47.236825 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:47.237088 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:47.238466 (+ 1378us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:47.238871 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.240619 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.243995 (+ 3376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.245415 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.246405 (+ 990us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.246553 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:47.246647 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 01:13:47.246802 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1996,"thread_start_us":1828,"threads_started":1}
I20260806 01:13:47.248006 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.235419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:35167 (local address 127.0.0.1:54026)
0806 01:13:47.236073 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:47.236385 (+ 312us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:47.236559 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:47.238295 (+ 1736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:47.238299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:47.238323 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:47.238788 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.238803 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.240801 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.240817 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.243553 (+ 2736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.243567 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.245771 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.245777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.245937 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.246033 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 01:13:47.246107 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 01:13:47.246266 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":138,"thread_start_us":351,"threads_started":1}
I20260806 01:13:47.251175 29763 tablet_service.cc:1467] Processing CreateTablet for tablet 1e9fe28c5a4347bfab5aa11d1efbbbcf (DEFAULT_TABLE table=test-table [id=fa06822b77e24ea58c69a4bfdcc72913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:47.251173 29562 tablet_service.cc:1467] Processing CreateTablet for tablet 1e9fe28c5a4347bfab5aa11d1efbbbcf (DEFAULT_TABLE table=test-table [id=fa06822b77e24ea58c69a4bfdcc72913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:47.251292 29353 tablet_service.cc:1467] Processing CreateTablet for tablet 1e9fe28c5a4347bfab5aa11d1efbbbcf (DEFAULT_TABLE table=test-table [id=fa06822b77e24ea58c69a4bfdcc72913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:47.252516 29763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e9fe28c5a4347bfab5aa11d1efbbbcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:47.252539 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e9fe28c5a4347bfab5aa11d1efbbbcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:47.252542 29353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e9fe28c5a4347bfab5aa11d1efbbbcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:47.266083 29908 tablet_bootstrap.cc:492] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af: Bootstrap starting.
I20260806 01:13:47.270100 29909 tablet_bootstrap.cc:492] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4: Bootstrap starting.
I20260806 01:13:47.270923 29908 tablet_bootstrap.cc:654] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:47.271006 29910 tablet_bootstrap.cc:492] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20: Bootstrap starting.
I20260806 01:13:47.272583 29908 log.cc:824] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:47.274021 29909 tablet_bootstrap.cc:654] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:47.274118 29910 tablet_bootstrap.cc:654] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:47.275264 29910 log.cc:824] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:47.275429 29909 log.cc:824] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:47.276528 29908 tablet_bootstrap.cc:492] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af: No bootstrap required, opened a new log
I20260806 01:13:47.276993 29908 ts_tablet_manager.cc:1397] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260806 01:13:47.278535 29910 tablet_bootstrap.cc:492] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20: No bootstrap required, opened a new log
I20260806 01:13:47.278874 29910 ts_tablet_manager.cc:1397] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 01:13:47.292321 29910 raft_consensus.cc:348] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } }
I20260806 01:13:47.292711 29909 tablet_bootstrap.cc:492] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4: No bootstrap required, opened a new log
I20260806 01:13:47.292953 29910 raft_consensus.cc:374] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:47.293076 29909 ts_tablet_manager.cc:1397] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4: Time spent bootstrapping tablet: real 0.023s user 0.015s sys 0.004s
I20260806 01:13:47.293118 29910 raft_consensus.cc:729] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c409779bcd184007bb77e07b0881fb20, State: Initialized, Role: FOLLOWER
I20260806 01:13:47.293933 29910 consensus_queue.cc:260] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } }
I20260806 01:13:47.294763 29908 raft_consensus.cc:348] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } }
I20260806 01:13:47.295204 29908 raft_consensus.cc:374] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:47.295305 29908 raft_consensus.cc:729] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec66fe008304da9a2e4a55e13a945af, State: Initialized, Role: FOLLOWER
I20260806 01:13:47.296006 29908 consensus_queue.cc:260] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } }
I20260806 01:13:47.302386 29910 ts_tablet_manager.cc:1428] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20: Time spent starting tablet: real 0.023s user 0.008s sys 0.014s
I20260806 01:13:47.304385 29908 ts_tablet_manager.cc:1428] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af: Time spent starting tablet: real 0.027s user 0.010s sys 0.012s
I20260806 01:13:47.304435 29881 heartbeater.cc:503] Master 127.25.236.190:37343 was elected leader, sending a full tablet report...
I20260806 01:13:47.307840 29909 raft_consensus.cc:348] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } }
I20260806 01:13:47.308386 29909 raft_consensus.cc:374] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:47.308542 29909 raft_consensus.cc:729] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048d166145f0424cbd94bca72db483b4, State: Initialized, Role: FOLLOWER
I20260806 01:13:47.309211 29909 consensus_queue.cc:260] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } }
I20260806 01:13:47.310500 29667 heartbeater.cc:503] Master 127.25.236.190:37343 was elected leader, sending a full tablet report...
I20260806 01:13:47.311497 29909 ts_tablet_manager.cc:1428] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
W20260806 01:13:47.390689 29882 tablet.cc:2369] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:13:47.482048 29464 tablet.cc:2369] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:13:47.555904 29668 tablet.cc:2369] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:13:47.607277 29915 raft_consensus.cc:482] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:13:47.607625 29915 raft_consensus.cc:504] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } }
I20260806 01:13:47.609323 29915 leader_election.cc:288] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c409779bcd184007bb77e07b0881fb20 (127.25.236.129:35167), 048d166145f0424cbd94bca72db483b4 (127.25.236.130:36681)
I20260806 01:13:47.611819 29914 raft_consensus.cc:482] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:13:47.612174 29914 raft_consensus.cc:504] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } }
I20260806 01:13:47.614221 29914 leader_election.cc:288] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229), 048d166145f0424cbd94bca72db483b4 (127.25.236.130:36681)
I20260806 01:13:47.619206 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.609839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:32939 (local address 127.25.236.129:35167)
0806 01:13:47.610055 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:47.610061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:47.610762 (+ 701us) server_negotiation.cc:407] Connection header received
0806 01:13:47.610959 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:47.610971 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:47.611030 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:47.611128 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:47.612006 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.613144 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.615650 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.617427 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.617599 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.617700 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:47.618623 (+ 923us) server_negotiation.cc:299] Negotiation successful
0806 01:13:47.618825 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:13:47.620376 29917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.609861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:35167 (local address 127.25.236.131:32939)
0806 01:13:47.610649 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:47.610664 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:47.610812 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:47.611146 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:47.611151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:47.611176 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:47.611808 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.611822 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.613299 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.613310 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.615445 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.615460 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.617971 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.617980 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.618358 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.618413 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 01:13:47.619585 (+ 1172us) client_negotiation.cc:240] Negotiation successful
0806 01:13:47.619828 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":48,"thread_start_us":533,"threads_started":1}
I20260806 01:13:47.621199 29907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.609894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:53971 (local address 127.25.236.130:36681)
0806 01:13:47.610119 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:47.610127 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:47.610147 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:13:47.610617 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:47.610624 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:47.610721 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:47.610841 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:47.611721 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.612873 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.615457 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.616722 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.616881 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.616972 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:47.620802 (+ 3830us) server_negotiation.cc:299] Negotiation successful
0806 01:13:47.620941 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 01:13:47.621297 29705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.609659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:36681 (local address 127.25.236.131:53971)
0806 01:13:47.610000 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:47.610023 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:47.610316 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:47.610859 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:47.610864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:47.610893 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:47.611434 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.611454 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.613026 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.613031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.615247 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.615264 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.618018 (+ 2754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.618025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.620635 (+ 2610us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.620677 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 01:13:47.620768 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 01:13:47.620913 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":2466}
I20260806 01:13:47.621639 29396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9fe28c5a4347bfab5aa11d1efbbbcf" candidate_uuid: "eec66fe008304da9a2e4a55e13a945af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c409779bcd184007bb77e07b0881fb20" is_pre_election: true
I20260806 01:13:47.623322 29591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9fe28c5a4347bfab5aa11d1efbbbcf" candidate_uuid: "eec66fe008304da9a2e4a55e13a945af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048d166145f0424cbd94bca72db483b4" is_pre_election: true
I20260806 01:13:47.622246 29396 raft_consensus.cc:2454] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eec66fe008304da9a2e4a55e13a945af in term 0.
I20260806 01:13:47.624004 29591 raft_consensus.cc:2454] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eec66fe008304da9a2e4a55e13a945af in term 0.
I20260806 01:13:47.624823 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.614517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:49339 (local address 127.25.236.131:33229)
0806 01:13:47.614722 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:47.614728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:47.615540 (+ 812us) server_negotiation.cc:407] Connection header received
0806 01:13:47.615796 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:47.615801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:47.615872 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:47.615928 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:47.616816 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.618067 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.621573 (+ 3506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.623288 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.623459 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.623557 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:47.624294 (+ 737us) server_negotiation.cc:299] Negotiation successful
0806 01:13:47.624464 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":67}
I20260806 01:13:47.625012 29696 leader_election.cc:302] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c409779bcd184007bb77e07b0881fb20, eec66fe008304da9a2e4a55e13a945af; no voters:
I20260806 01:13:47.625485 29915 raft_consensus.cc:2781] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:13:47.625674 29915 raft_consensus.cc:482] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:13:47.625818 29915 raft_consensus.cc:3037] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:47.626358 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.614761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:33229 (local address 127.25.236.129:49339)
0806 01:13:47.615452 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:47.615470 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:47.615651 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:47.616120 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:47.616124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:47.616149 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:47.616641 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.616654 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.618724 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.618733 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.621375 (+ 2642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.621395 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.623788 (+ 2393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.623796 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.624047 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.624086 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 01:13:47.625008 (+ 922us) client_negotiation.cc:240] Negotiation successful
0806 01:13:47.625194 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":359,"threads_started":1}
I20260806 01:13:47.627329 29806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9fe28c5a4347bfab5aa11d1efbbbcf" candidate_uuid: "c409779bcd184007bb77e07b0881fb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec66fe008304da9a2e4a55e13a945af" is_pre_election: true
I20260806 01:13:47.629137 29915 raft_consensus.cc:504] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } }
I20260806 01:13:47.629635 29806 raft_consensus.cc:2379] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c409779bcd184007bb77e07b0881fb20 in current term 1: Already voted for candidate eec66fe008304da9a2e4a55e13a945af in this term.
I20260806 01:13:47.630183 29907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.617104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:55633 (local address 127.25.236.130:36681)
0806 01:13:47.621739 (+ 4635us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:47.621746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:47.622080 (+ 334us) server_negotiation.cc:407] Connection header received
0806 01:13:47.622397 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:47.622403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:47.622510 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:47.622589 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:47.623453 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.625878 (+ 2425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.628267 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.629100 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.629305 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.629391 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:47.629698 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 01:13:47.629830 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4537,"thread_start_us":10602,"threads_started":1}
I20260806 01:13:47.630420 29919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.615531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:36681 (local address 127.25.236.129:55633)
0806 01:13:47.621964 (+ 6433us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:47.621991 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:47.622190 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:47.622805 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:47.622810 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:47.622841 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:47.623236 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.623250 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.626022 (+ 2772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.626028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.628099 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.628114 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.629343 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.629350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.629528 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.629552 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:13:47.629671 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 01:13:47.629803 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6289,"thread_start_us":8711,"threads_started":1}
I20260806 01:13:47.630393 29396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9fe28c5a4347bfab5aa11d1efbbbcf" candidate_uuid: "eec66fe008304da9a2e4a55e13a945af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c409779bcd184007bb77e07b0881fb20"
I20260806 01:13:47.630667 29396 raft_consensus.cc:3037] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:47.631136 29591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9fe28c5a4347bfab5aa11d1efbbbcf" candidate_uuid: "c409779bcd184007bb77e07b0881fb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048d166145f0424cbd94bca72db483b4" is_pre_election: true
I20260806 01:13:47.631415 29591 raft_consensus.cc:2454] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c409779bcd184007bb77e07b0881fb20 in term 0.
I20260806 01:13:47.631932 29279 leader_election.cc:302] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 048d166145f0424cbd94bca72db483b4, c409779bcd184007bb77e07b0881fb20; no voters: eec66fe008304da9a2e4a55e13a945af
I20260806 01:13:47.633919 29396 raft_consensus.cc:2454] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eec66fe008304da9a2e4a55e13a945af in term 1.
I20260806 01:13:47.634110 29914 raft_consensus.cc:2741] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 01:13:47.634449 29696 leader_election.cc:302] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c409779bcd184007bb77e07b0881fb20, eec66fe008304da9a2e4a55e13a945af; no voters:
I20260806 01:13:47.634964 29591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9fe28c5a4347bfab5aa11d1efbbbcf" candidate_uuid: "eec66fe008304da9a2e4a55e13a945af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048d166145f0424cbd94bca72db483b4"
I20260806 01:13:47.635202 29591 raft_consensus.cc:3037] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:47.634454 29915 leader_election.cc:288] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [CANDIDATE]: Term 1 election: Requested vote from peers c409779bcd184007bb77e07b0881fb20 (127.25.236.129:35167), 048d166145f0424cbd94bca72db483b4 (127.25.236.130:36681)
I20260806 01:13:47.635463 29922 raft_consensus.cc:2781] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:47.635682 29922 raft_consensus.cc:686] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 1 LEADER]: Becoming Leader. State: Replica: eec66fe008304da9a2e4a55e13a945af, State: Running, Role: LEADER
I20260806 01:13:47.636215 29922 consensus_queue.cc:237] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } }
I20260806 01:13:47.637780 29591 raft_consensus.cc:2454] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eec66fe008304da9a2e4a55e13a945af in term 1.
I20260806 01:13:47.640548 29146 catalog_manager.cc:5495] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af reported cstate change: term changed from 0 to 1, leader changed from <none> to eec66fe008304da9a2e4a55e13a945af (127.25.236.131). New cstate: current_term: 1 leader_uuid: "eec66fe008304da9a2e4a55e13a945af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } health_report { overall_health: UNKNOWN } } }
Aug 06 01:13:47 dist-test-slave-7nm7 krb5kdc[29076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978825, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:47.735194 29931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.720537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:37343 (local address 127.0.0.1:41030)
0806 01:13:47.721247 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:47.721270 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:47.721402 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:47.721785 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:47.721788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:47.722614 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:47.722870 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.722879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.723987 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.723992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.725136 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:47.725148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.725935 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.725945 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:47.726069 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.726857 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:47.726877 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:47.729224 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:47.732639 (+ 3415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:47.732650 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:47.732655 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:47.733097 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:47.733554 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:47.733558 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:47.733560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:47.733728 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:47.734272 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:47.734285 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:47.734485 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 01:13:47.734629 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 01:13:47.734894 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":355,"threads_started":1}
I20260806 01:13:47.735571 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:47.720621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41030 (local address 127.25.236.190:37343)
0806 01:13:47.721348 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:47.721352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:47.721376 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:13:47.721551 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:47.721555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:47.721624 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:47.721719 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:47.723041 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.723840 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.725293 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:47.725734 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:47.725842 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:47.729461 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:47.729466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:47.729469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:47.729499 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:47.732437 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:47.733265 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:47.733271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:47.733273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:47.733433 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:47.733824 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:47.733827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:47.733829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:47.734115 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:47.734273 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:47.734708 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 01:13:47.735099 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":598,"thread_start_us":439,"threads_started":1}
I20260806 01:13:47.741067 29146 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:41030:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 01:13:47.742458 29146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:13:47.750473 29763 tablet_service.cc:1467] Processing CreateTablet for tablet 759e1d996b5b4ea5a02a69216a848a38 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=afed94f119a8427fb6e9c096868f72f8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:13:47.750301 29353 tablet_service.cc:1467] Processing CreateTablet for tablet 759e1d996b5b4ea5a02a69216a848a38 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=afed94f119a8427fb6e9c096868f72f8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:13:47.750535 29562 tablet_service.cc:1467] Processing CreateTablet for tablet 759e1d996b5b4ea5a02a69216a848a38 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=afed94f119a8427fb6e9c096868f72f8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:13:47.751315 29763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759e1d996b5b4ea5a02a69216a848a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:47.751322 29353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759e1d996b5b4ea5a02a69216a848a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:47.751322 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759e1d996b5b4ea5a02a69216a848a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:47.757191 29908 tablet_bootstrap.cc:492] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af: Bootstrap starting.
I20260806 01:13:47.758816 29909 tablet_bootstrap.cc:492] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4: Bootstrap starting.
I20260806 01:13:47.759089 29908 tablet_bootstrap.cc:654] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:47.759141 29910 tablet_bootstrap.cc:492] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20: Bootstrap starting.
I20260806 01:13:47.760825 29910 tablet_bootstrap.cc:654] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:47.760923 29909 tablet_bootstrap.cc:654] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:47.764637 29909 tablet_bootstrap.cc:492] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4: No bootstrap required, opened a new log
I20260806 01:13:47.764864 29909 ts_tablet_manager.cc:1397] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 01:13:47.766034 29909 raft_consensus.cc:348] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:47.766538 29909 raft_consensus.cc:374] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:47.766686 29909 raft_consensus.cc:729] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048d166145f0424cbd94bca72db483b4, State: Initialized, Role: FOLLOWER
I20260806 01:13:47.767140 29909 consensus_queue.cc:260] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:47.767974 29909 ts_tablet_manager.cc:1428] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 01:13:47.767800 29916 tablet_replica.cc:439] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } }
I20260806 01:13:47.768234 29916 tablet_replica.cc:442] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:47.771420 29908 tablet_bootstrap.cc:492] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af: No bootstrap required, opened a new log
I20260806 01:13:47.771597 29908 ts_tablet_manager.cc:1397] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20260806 01:13:47.771970 29910 tablet_bootstrap.cc:492] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20: No bootstrap required, opened a new log
I20260806 01:13:47.772382 29910 ts_tablet_manager.cc:1397] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260806 01:13:47.772759 29908 raft_consensus.cc:348] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:47.773015 29908 raft_consensus.cc:374] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:47.773133 29908 raft_consensus.cc:729] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec66fe008304da9a2e4a55e13a945af, State: Initialized, Role: FOLLOWER
I20260806 01:13:47.773264 29910 raft_consensus.cc:348] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:47.773540 29910 raft_consensus.cc:374] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:47.773459 29908 consensus_queue.cc:260] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:47.773641 29910 raft_consensus.cc:729] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c409779bcd184007bb77e07b0881fb20, State: Initialized, Role: FOLLOWER
I20260806 01:13:47.773914 29910 consensus_queue.cc:260] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:47.773922 29922 tablet_replica.cc:439] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } }
I20260806 01:13:47.774214 29922 tablet_replica.cc:442] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:47.774389 29914 tablet_replica.cc:439] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } }
I20260806 01:13:47.774647 29914 tablet_replica.cc:442] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:47.775617 29910 ts_tablet_manager.cc:1428] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 01:13:47.775488 29908 ts_tablet_manager.cc:1428] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260806 01:13:48.024199 29916 raft_consensus.cc:482] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:13:48.024479 29916 raft_consensus.cc:504] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:48.026265 29916 leader_election.cc:288] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c409779bcd184007bb77e07b0881fb20 (127.25.236.129:35167), eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229)
I20260806 01:13:48.034452 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.027088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:40233 (local address 127.25.236.131:33229)
0806 01:13:48.027500 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:48.027510 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:48.028256 (+ 746us) server_negotiation.cc:407] Connection header received
0806 01:13:48.028464 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:48.028470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:48.028543 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:48.028651 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:48.029208 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.029979 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.032029 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.033052 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.033425 (+ 373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.033532 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:48.033907 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 01:13:48.034044 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 01:13:48.034466 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.026750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:44143 (local address 127.25.236.129:35167)
0806 01:13:48.026977 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:48.026985 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:48.027809 (+ 824us) server_negotiation.cc:407] Connection header received
0806 01:13:48.028069 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:48.028082 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:48.028151 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:48.028260 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:13:48.028890 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.029697 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.032064 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.032996 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.033239 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.033347 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:48.033824 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 01:13:48.033977 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 01:13:48.035053 29938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.027055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:33229 (local address 127.25.236.130:40233)
0806 01:13:48.028133 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:48.028149 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:48.028320 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:48.028673 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:48.028677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:48.028700 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:48.029059 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.029069 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.030146 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.030154 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.031880 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.031893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.033575 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.033582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.033741 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.033763 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 01:13:48.033880 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 01:13:48.034023 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":53,"thread_start_us":750,"threads_started":1}
I20260806 01:13:48.035215 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.026740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:35167 (local address 127.25.236.130:44143)
0806 01:13:48.027671 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:48.027690 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:48.027901 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:48.028292 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:48.028297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:48.028329 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:13:48.028688 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.028698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.029857 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.029863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.031891 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.031905 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.033495 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.033501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.033632 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.033663 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 01:13:48.033796 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 01:13:48.033956 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"mutex_wait_us":45,"thread_start_us":489,"threads_started":1}
I20260806 01:13:48.036221 29806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759e1d996b5b4ea5a02a69216a848a38" candidate_uuid: "048d166145f0424cbd94bca72db483b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec66fe008304da9a2e4a55e13a945af" is_pre_election: true
I20260806 01:13:48.036197 29396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759e1d996b5b4ea5a02a69216a848a38" candidate_uuid: "048d166145f0424cbd94bca72db483b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c409779bcd184007bb77e07b0881fb20" is_pre_election: true
I20260806 01:13:48.036530 29806 raft_consensus.cc:2454] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 048d166145f0424cbd94bca72db483b4 in term 0.
I20260806 01:13:48.036580 29396 raft_consensus.cc:2454] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 048d166145f0424cbd94bca72db483b4 in term 0.
I20260806 01:13:48.037200 29495 leader_election.cc:302] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048d166145f0424cbd94bca72db483b4, eec66fe008304da9a2e4a55e13a945af; no voters:
I20260806 01:13:48.037621 29916 raft_consensus.cc:2781] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:13:48.037761 29916 raft_consensus.cc:482] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:13:48.037855 29916 raft_consensus.cc:3037] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:48.040457 29916 raft_consensus.cc:504] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:48.041082 29916 leader_election.cc:288] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [CANDIDATE]: Term 1 election: Requested vote from peers c409779bcd184007bb77e07b0881fb20 (127.25.236.129:35167), eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229)
I20260806 01:13:48.041646 29806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759e1d996b5b4ea5a02a69216a848a38" candidate_uuid: "048d166145f0424cbd94bca72db483b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec66fe008304da9a2e4a55e13a945af"
I20260806 01:13:48.041638 29396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759e1d996b5b4ea5a02a69216a848a38" candidate_uuid: "048d166145f0424cbd94bca72db483b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c409779bcd184007bb77e07b0881fb20"
I20260806 01:13:48.041857 29806 raft_consensus.cc:3037] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:48.041944 29396 raft_consensus.cc:3037] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:48.044100 29806 raft_consensus.cc:2454] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 048d166145f0424cbd94bca72db483b4 in term 1.
I20260806 01:13:48.044376 29396 raft_consensus.cc:2454] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 048d166145f0424cbd94bca72db483b4 in term 1.
I20260806 01:13:48.044657 29495 leader_election.cc:302] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048d166145f0424cbd94bca72db483b4, eec66fe008304da9a2e4a55e13a945af; no voters:
I20260806 01:13:48.045063 29916 raft_consensus.cc:2781] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:48.045665 29916 raft_consensus.cc:686] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [term 1 LEADER]: Becoming Leader. State: Replica: 048d166145f0424cbd94bca72db483b4, State: Running, Role: LEADER
I20260806 01:13:48.046329 29916 consensus_queue.cc:237] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:48.049994 29939 tablet_replica.cc:439] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4: TxnStatusTablet state changed. Reason: New leader 048d166145f0424cbd94bca72db483b4. Latest consensus state: current_term: 1 leader_uuid: "048d166145f0424cbd94bca72db483b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } }
I20260806 01:13:48.050709 29939 tablet_replica.cc:442] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:13:48.051530 29942 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 01:13:48.051867 29942 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 01:13:48.051982 29147 catalog_manager.cc:5495] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 048d166145f0424cbd94bca72db483b4 (127.25.236.130). New cstate: current_term: 1 leader_uuid: "048d166145f0424cbd94bca72db483b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } health_report { overall_health: UNKNOWN } } }
I20260806 01:13:48.056823 29922 consensus_queue.cc:1035] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [LEADER]: Connected to new peer: Peer: permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:48.069180 29922 consensus_queue.cc:1035] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [LEADER]: Connected to new peer: Peer: permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:48.146119 29956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.135722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47724 (local address 127.25.236.130:36681)
0806 01:13:48.139202 (+ 3480us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:48.139212 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:48.139233 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:13:48.139340 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:48.139346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:48.139460 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:48.139625 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:48.140103 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.141240 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.143295 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.143981 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.144111 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.144679 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:48.145400 (+ 721us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:48.145548 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:48.145705 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 01:13:48.145793 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3333,"thread_start_us":3180,"threads_started":1}
I20260806 01:13:48.146160 29931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.135673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:36681 (local address 127.0.0.1:47724)
0806 01:13:48.135913 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:48.135939 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:48.136085 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:48.139679 (+ 3594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:48.139685 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:48.139701 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:13:48.139923 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.139935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.141387 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.141394 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.143132 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.143149 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.144170 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.144176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.144368 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.144454 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:13:48.145520 (+ 1066us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:13:48.145545 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 01:13:48.145642 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 01:13:48.145812 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 01:13:48.155368 29806 raft_consensus.cc:1261] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Refusing update from remote peer 048d166145f0424cbd94bca72db483b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:48.155531 29396 raft_consensus.cc:1261] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Refusing update from remote peer 048d166145f0424cbd94bca72db483b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:48.156363 29916 consensus_queue.cc:1035] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:48.157101 29939 consensus_queue.cc:1035] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:48.163540 29922 tablet_replica.cc:439] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af: TxnStatusTablet state changed. Reason: New leader 048d166145f0424cbd94bca72db483b4. Latest consensus state: current_term: 1 leader_uuid: "048d166145f0424cbd94bca72db483b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } }
I20260806 01:13:48.163929 29922 tablet_replica.cc:442] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:48.168046 29952 mvcc.cc:205] Tried to move back new op lower bound from 7315369280114176000 to 7315369279681945600. Current Snapshot: MvccSnapshot[applied={T|T < 7315369280114176000}]
I20260806 01:13:48.175969 29922 tablet_replica.cc:439] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "048d166145f0424cbd94bca72db483b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } }
I20260806 01:13:48.176581 29922 tablet_replica.cc:442] T 759e1d996b5b4ea5a02a69216a848a38 P eec66fe008304da9a2e4a55e13a945af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:48.178002 29914 tablet_replica.cc:439] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20: TxnStatusTablet state changed. Reason: New leader 048d166145f0424cbd94bca72db483b4. Latest consensus state: current_term: 1 leader_uuid: "048d166145f0424cbd94bca72db483b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } }
I20260806 01:13:48.178434 29914 tablet_replica.cc:442] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:48.178686 29939 tablet_replica.cc:439] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "048d166145f0424cbd94bca72db483b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } }
I20260806 01:13:48.178980 29939 tablet_replica.cc:442] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:13:48.180195 29939 tablet_replica.cc:439] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "048d166145f0424cbd94bca72db483b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } }
I20260806 01:13:48.180536 29939 tablet_replica.cc:442] T 759e1d996b5b4ea5a02a69216a848a38 P 048d166145f0424cbd94bca72db483b4: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:13:48.197324 29914 tablet_replica.cc:439] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "048d166145f0424cbd94bca72db483b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } } }
I20260806 01:13:48.197638 29914 tablet_replica.cc:442] T 759e1d996b5b4ea5a02a69216a848a38 P c409779bcd184007bb77e07b0881fb20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:13:48.207962 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.201465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57546 (local address 127.25.236.131:33229)
0806 01:13:48.201783 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:48.201791 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:48.201955 (+ 164us) server_negotiation.cc:407] Connection header received
0806 01:13:48.202298 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:48.202304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:48.202420 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:48.202575 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:48.203041 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.203773 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.205400 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.205982 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.206182 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.206677 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:48.207223 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:48.207390 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:48.207471 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 01:13:48.207555 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 01:13:48.227937 29956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.220822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:40807 (local address 127.25.236.130:36681)
0806 01:13:48.221086 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:48.221096 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:48.221689 (+ 593us) server_negotiation.cc:407] Connection header received
0806 01:13:48.221896 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:48.221901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:48.221973 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:48.222066 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:48.222673 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.223516 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.225619 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.226477 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.226599 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.227216 (+ 617us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:48.227343 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:48.227440 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:48.227613 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 01:13:48.227684 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:13:48.228116 29977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.220831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:36681 (local address 127.25.236.131:40807)
0806 01:13:48.221570 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:48.221590 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:48.221750 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:48.222104 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:48.222109 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:48.222122 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:13:48.222501 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.222514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.223652 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.223658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.225344 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.225359 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.226676 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.226683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.226872 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.226944 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:13:48.227468 (+ 524us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:13:48.227487 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 01:13:48.227576 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 01:13:48.227727 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":403,"threads_started":1}
I20260806 01:13:48.270498 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.262983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:40031 (local address 127.25.236.131:33229)
0806 01:13:48.263419 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:48.263426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:48.263997 (+ 571us) server_negotiation.cc:407] Connection header received
0806 01:13:48.264283 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:48.264288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:48.264367 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:48.264464 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:48.265014 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.266082 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.268191 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.268841 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.269007 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.269555 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:48.269710 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:48.269832 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:48.269986 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 01:13:48.270076 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284}
I20260806 01:13:48.270676 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.263074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:33229 (local address 127.25.236.130:40031)
0806 01:13:48.263888 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:48.263914 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:48.264140 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:48.264476 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:48.264481 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:48.264492 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:13:48.264851 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.264860 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.266221 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.266228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.268022 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.268039 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.269026 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.269033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.269212 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.269296 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:13:48.269832 (+ 536us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:13:48.269853 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 01:13:48.269959 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 01:13:48.270134 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":922,"threads_started":1}
I20260806 01:13:48.324651 29146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:13:48.324959 29146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:13:48.329027 29146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d: Sending DeleteTablet for 3 replicas of tablet 1e9fe28c5a4347bfab5aa11d1efbbbcf
I20260806 01:13:48.330219 29763 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e9fe28c5a4347bfab5aa11d1efbbbcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:48 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:57532
I20260806 01:13:48.330346 29353 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e9fe28c5a4347bfab5aa11d1efbbbcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:48 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:54026
I20260806 01:13:48.331805 29562 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e9fe28c5a4347bfab5aa11d1efbbbcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:48 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:47712
I20260806 01:13:48.339890 29147 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:13:48.340580 29147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:13:48.342924 29995 tablet_replica.cc:330] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4: stopping tablet replica
I20260806 01:13:48.343564 29995 raft_consensus.cc:2229] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:13:48.344043 29995 raft_consensus.cc:2258] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:48.348934 29992 tablet_replica.cc:330] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af: stopping tablet replica
I20260806 01:13:48.349447 29992 raft_consensus.cc:2229] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:13:48.351023 29350 tablet_service.cc:1467] Processing CreateTablet for tablet f82fe438aaa64365b8a36d18c0ae1646 (DEFAULT_TABLE table=test-table [id=f764c1c0068d4a38ba374ef67ba11c6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:48.351717 29350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82fe438aaa64365b8a36d18c0ae1646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:48.351686 29562 tablet_service.cc:1467] Processing CreateTablet for tablet f82fe438aaa64365b8a36d18c0ae1646 (DEFAULT_TABLE table=test-table [id=f764c1c0068d4a38ba374ef67ba11c6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:48.352203 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82fe438aaa64365b8a36d18c0ae1646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:48.355049 29995 ts_tablet_manager.cc:1905] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:48.363981 29997 tablet_bootstrap.cc:492] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4: Bootstrap starting.
I20260806 01:13:48.367076 29759 tablet_service.cc:1467] Processing CreateTablet for tablet f82fe438aaa64365b8a36d18c0ae1646 (DEFAULT_TABLE table=test-table [id=f764c1c0068d4a38ba374ef67ba11c6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:48.367810 29759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82fe438aaa64365b8a36d18c0ae1646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:48.369059 29994 tablet_replica.cc:330] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20: stopping tablet replica
I20260806 01:13:48.369632 29994 raft_consensus.cc:2229] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:13:48.370103 29994 raft_consensus.cc:2258] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:48.370658 29995 ts_tablet_manager.cc:1918] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:13:48.370728 29996 tablet_bootstrap.cc:492] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20: Bootstrap starting.
I20260806 01:13:48.371191 29995 log.cc:1196] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/1e9fe28c5a4347bfab5aa11d1efbbbcf
I20260806 01:13:48.371933 29995 ts_tablet_manager.cc:1939] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P 048d166145f0424cbd94bca72db483b4: Deleting consensus metadata
I20260806 01:13:48.373093 29994 ts_tablet_manager.cc:1905] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:48.375125 29122 catalog_manager.cc:4841] TS 048d166145f0424cbd94bca72db483b4 (127.25.236.130:36681): tablet 1e9fe28c5a4347bfab5aa11d1efbbbcf (table test-table [id=fa06822b77e24ea58c69a4bfdcc72913]) successfully deleted
I20260806 01:13:48.376969 29992 raft_consensus.cc:2258] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:48.377374 29998 tablet_bootstrap.cc:492] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af: Bootstrap starting.
I20260806 01:13:48.379171 29992 ts_tablet_manager.cc:1905] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:48.379793 29996 tablet_bootstrap.cc:654] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:48.381366 29998 tablet_bootstrap.cc:654] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:48.386415 29997 tablet_bootstrap.cc:654] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:48.391847 29996 tablet_bootstrap.cc:492] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20: No bootstrap required, opened a new log
I20260806 01:13:48.392081 29996 ts_tablet_manager.cc:1397] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.000s
I20260806 01:13:48.392949 29996 raft_consensus.cc:348] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:48.393224 29996 raft_consensus.cc:374] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:48.393316 29996 raft_consensus.cc:729] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c409779bcd184007bb77e07b0881fb20, State: Initialized, Role: FOLLOWER
I20260806 01:13:48.393610 29996 consensus_queue.cc:260] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:48.394670 29994 ts_tablet_manager.cc:1918] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:13:48.396965 29998 tablet_bootstrap.cc:492] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af: No bootstrap required, opened a new log
I20260806 01:13:48.397303 29994 log.cc:1196] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/1e9fe28c5a4347bfab5aa11d1efbbbcf
I20260806 01:13:48.397316 29998 ts_tablet_manager.cc:1397] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.000s
I20260806 01:13:48.398007 29994 ts_tablet_manager.cc:1939] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P c409779bcd184007bb77e07b0881fb20: Deleting consensus metadata
I20260806 01:13:48.398146 29996 ts_tablet_manager.cc:1428] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 01:13:48.398190 29998 raft_consensus.cc:348] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:48.398494 29998 raft_consensus.cc:374] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:48.398617 29998 raft_consensus.cc:729] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec66fe008304da9a2e4a55e13a945af, State: Initialized, Role: FOLLOWER
I20260806 01:13:48.398965 29998 consensus_queue.cc:260] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:48.399636 29998 ts_tablet_manager.cc:1428] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 01:13:48.401610 29992 ts_tablet_manager.cc:1918] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:13:48.401827 29992 log.cc:1196] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/1e9fe28c5a4347bfab5aa11d1efbbbcf
I20260806 01:13:48.402469 29992 ts_tablet_manager.cc:1939] T 1e9fe28c5a4347bfab5aa11d1efbbbcf P eec66fe008304da9a2e4a55e13a945af: Deleting consensus metadata
I20260806 01:13:48.404426 29122 catalog_manager.cc:4841] TS eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229): tablet 1e9fe28c5a4347bfab5aa11d1efbbbcf (table test-table [id=fa06822b77e24ea58c69a4bfdcc72913]) successfully deleted
I20260806 01:13:48.405593 29997 tablet_bootstrap.cc:492] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4: No bootstrap required, opened a new log
I20260806 01:13:48.405769 29997 ts_tablet_manager.cc:1397] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4: Time spent bootstrapping tablet: real 0.042s user 0.000s sys 0.009s
I20260806 01:13:48.406541 29121 catalog_manager.cc:4841] TS c409779bcd184007bb77e07b0881fb20 (127.25.236.129:35167): tablet 1e9fe28c5a4347bfab5aa11d1efbbbcf (table test-table [id=fa06822b77e24ea58c69a4bfdcc72913]) successfully deleted
I20260806 01:13:48.406689 29997 raft_consensus.cc:348] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:48.406992 29997 raft_consensus.cc:374] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:48.407104 29997 raft_consensus.cc:729] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048d166145f0424cbd94bca72db483b4, State: Initialized, Role: FOLLOWER
I20260806 01:13:48.407384 29997 consensus_queue.cc:260] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:48.409070 29997 ts_tablet_manager.cc:1428] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 01:13:48.642462 29914 raft_consensus.cc:482] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:13:48.642683 29914 raft_consensus.cc:504] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:48.643339 29914 leader_election.cc:288] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 048d166145f0424cbd94bca72db483b4 (127.25.236.130:36681), eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229)
I20260806 01:13:48.644025 29591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82fe438aaa64365b8a36d18c0ae1646" candidate_uuid: "c409779bcd184007bb77e07b0881fb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048d166145f0424cbd94bca72db483b4" is_pre_election: true
I20260806 01:13:48.644120 29806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82fe438aaa64365b8a36d18c0ae1646" candidate_uuid: "c409779bcd184007bb77e07b0881fb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec66fe008304da9a2e4a55e13a945af" is_pre_election: true
I20260806 01:13:48.644340 29591 raft_consensus.cc:2454] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c409779bcd184007bb77e07b0881fb20 in term 0.
I20260806 01:13:48.644487 29806 raft_consensus.cc:2454] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c409779bcd184007bb77e07b0881fb20 in term 0.
I20260806 01:13:48.644898 29279 leader_election.cc:302] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048d166145f0424cbd94bca72db483b4, c409779bcd184007bb77e07b0881fb20; no voters:
I20260806 01:13:48.645262 29914 raft_consensus.cc:2781] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:13:48.645506 29914 raft_consensus.cc:482] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:13:48.645640 29914 raft_consensus.cc:3037] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:48.649940 29914 raft_consensus.cc:504] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:48.650995 29914 leader_election.cc:288] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [CANDIDATE]: Term 1 election: Requested vote from peers 048d166145f0424cbd94bca72db483b4 (127.25.236.130:36681), eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229)
I20260806 01:13:48.651646 29591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82fe438aaa64365b8a36d18c0ae1646" candidate_uuid: "c409779bcd184007bb77e07b0881fb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048d166145f0424cbd94bca72db483b4"
I20260806 01:13:48.651834 29806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82fe438aaa64365b8a36d18c0ae1646" candidate_uuid: "c409779bcd184007bb77e07b0881fb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec66fe008304da9a2e4a55e13a945af"
I20260806 01:13:48.651988 29591 raft_consensus.cc:3037] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:48.652144 29806 raft_consensus.cc:3037] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:48.655637 29591 raft_consensus.cc:2454] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c409779bcd184007bb77e07b0881fb20 in term 1.
I20260806 01:13:48.656201 29279 leader_election.cc:302] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048d166145f0424cbd94bca72db483b4, c409779bcd184007bb77e07b0881fb20; no voters:
I20260806 01:13:48.656558 29914 raft_consensus.cc:2781] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:48.656828 29806 raft_consensus.cc:2454] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c409779bcd184007bb77e07b0881fb20 in term 1.
I20260806 01:13:48.657465 29914 raft_consensus.cc:686] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [term 1 LEADER]: Becoming Leader. State: Replica: c409779bcd184007bb77e07b0881fb20, State: Running, Role: LEADER
I20260806 01:13:48.658368 29914 consensus_queue.cc:237] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:48.662973 29147 catalog_manager.cc:5495] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 reported cstate change: term changed from 0 to 1, leader changed from <none> to c409779bcd184007bb77e07b0881fb20 (127.25.236.129). New cstate: current_term: 1 leader_uuid: "c409779bcd184007bb77e07b0881fb20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } health_report { overall_health: UNKNOWN } } }
I20260806 01:13:48.755867 30013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.747658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54028 (local address 127.25.236.129:35167)
0806 01:13:48.748863 (+ 1205us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:48.748868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:48.748897 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:13:48.748983 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:48.748987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:48.749092 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:48.749291 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:48.749811 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.751019 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.752444 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.752949 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.753096 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.753647 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:48.754634 (+ 987us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:48.754914 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:48.755089 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 01:13:48.755210 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1025,"thread_start_us":809,"threads_started":1}
I20260806 01:13:48.778728 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.770671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:37635 (local address 127.25.236.130:36681)
0806 01:13:48.771620 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:48.771633 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:48.772529 (+ 896us) server_negotiation.cc:407] Connection header received
0806 01:13:48.772828 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:48.772832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:48.772901 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:48.773011 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:48.774341 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.775161 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.776796 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.777206 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.777350 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.777829 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:48.777945 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:48.778063 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:48.778226 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 01:13:48.778354 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838,"thread_start_us":724,"threads_started":1}
I20260806 01:13:48.779074 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.770759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:36681 (local address 127.25.236.129:37635)
0806 01:13:48.772418 (+ 1659us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:48.772436 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:48.772666 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:48.773026 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:48.773030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:48.773042 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:13:48.773906 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.773918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.775378 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.775383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.776633 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.776646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.777405 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.777414 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.777556 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.777621 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:13:48.778064 (+ 443us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:13:48.778078 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 01:13:48.778200 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 01:13:48.778403 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1483,"thread_start_us":1174,"threads_started":1}
I20260806 01:13:48.791252 29591 raft_consensus.cc:1261] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Refusing update from remote peer c409779bcd184007bb77e07b0881fb20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:48.791464 29806 raft_consensus.cc:1261] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Refusing update from remote peer c409779bcd184007bb77e07b0881fb20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:48.792096 30002 consensus_queue.cc:1035] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:48.792610 29914 consensus_queue.cc:1035] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:48.797686 29949 mvcc.cc:205] Tried to move back new op lower bound from 7315369282721583104 to 7315369282189778944. Current Snapshot: MvccSnapshot[applied={T|T < 7315369282721583104}]
I20260806 01:13:48.811625 29947 mvcc.cc:205] Tried to move back new op lower bound from 7315369282721583104 to 7315369282189778944. Current Snapshot: MvccSnapshot[applied={T|T < 7315369282721583104}]
I20260806 01:13:48.834661 30022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.824144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:35167 (local address 127.25.236.130:49449)
0806 01:13:48.826760 (+ 2616us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:48.826782 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:48.826916 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:48.827563 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:48.827569 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:48.827584 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:13:48.827956 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.827972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.829976 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.829983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.832012 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:48.832031 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.832939 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.832946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:48.833244 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.833308 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:13:48.833858 (+ 550us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:13:48.833875 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 01:13:48.833982 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 01:13:48.834152 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2435,"thread_start_us":2216,"threads_started":1}
I20260806 01:13:48.834662 30013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:48.824710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:49449 (local address 127.25.236.129:35167)
0806 01:13:48.824958 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:48.824965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:48.826899 (+ 1934us) server_negotiation.cc:407] Connection header received
0806 01:13:48.827067 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:48.827074 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:48.827178 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:48.827288 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:48.828311 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.829813 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.832234 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:48.832716 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:48.832888 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:48.833581 (+ 693us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:48.833725 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:48.833806 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:48.834017 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 01:13:48.834110 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
W20260806 01:13:48.836230 29949 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 01:13:50.463181 29147 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:13:50.463693 29147 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:13:50.468257 29147 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d: Sending DeleteTablet for 3 replicas of tablet f82fe438aaa64365b8a36d18c0ae1646
I20260806 01:13:50.469138 29353 tablet_service.cc:1514] Processing DeleteTablet for tablet f82fe438aaa64365b8a36d18c0ae1646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:50 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:54026
I20260806 01:13:50.469313 29562 tablet_service.cc:1514] Processing DeleteTablet for tablet f82fe438aaa64365b8a36d18c0ae1646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:50 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:47712
I20260806 01:13:50.469501 29759 tablet_service.cc:1514] Processing DeleteTablet for tablet f82fe438aaa64365b8a36d18c0ae1646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:50 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:57532
I20260806 01:13:50.485028 30060 tablet_replica.cc:330] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af: stopping tablet replica
I20260806 01:13:50.485059 30058 tablet_replica.cc:330] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4: stopping tablet replica
I20260806 01:13:50.485320 30060 raft_consensus.cc:2229] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:13:50.485354 30058 raft_consensus.cc:2229] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:13:50.485641 30058 raft_consensus.cc:2258] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:50.485692 30060 raft_consensus.cc:2258] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:50.487422 30057 tablet_replica.cc:330] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20: stopping tablet replica
I20260806 01:13:50.487686 30060 ts_tablet_manager.cc:1905] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:50.487699 30057 raft_consensus.cc:2229] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:13:50.488011 30058 ts_tablet_manager.cc:1905] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:50.488534 30057 raft_consensus.cc:2258] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:50.489686 30057 ts_tablet_manager.cc:1905] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:50.495549 30058 ts_tablet_manager.cc:1918] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:13:50.495565 30060 ts_tablet_manager.cc:1918] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:13:50.495707 30060 log.cc:1196] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/f82fe438aaa64365b8a36d18c0ae1646
I20260806 01:13:50.495844 30058 log.cc:1196] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/f82fe438aaa64365b8a36d18c0ae1646
I20260806 01:13:50.496356 30060 ts_tablet_manager.cc:1939] T f82fe438aaa64365b8a36d18c0ae1646 P eec66fe008304da9a2e4a55e13a945af: Deleting consensus metadata
I20260806 01:13:50.496534 30058 ts_tablet_manager.cc:1939] T f82fe438aaa64365b8a36d18c0ae1646 P 048d166145f0424cbd94bca72db483b4: Deleting consensus metadata
I20260806 01:13:50.497550 30057 ts_tablet_manager.cc:1918] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:13:50.497695 30057 log.cc:1196] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/f82fe438aaa64365b8a36d18c0ae1646
I20260806 01:13:50.497643 29122 catalog_manager.cc:4841] TS eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229): tablet f82fe438aaa64365b8a36d18c0ae1646 (table test-table [id=f764c1c0068d4a38ba374ef67ba11c6a]) successfully deleted
I20260806 01:13:50.497963 29122 catalog_manager.cc:4841] TS 048d166145f0424cbd94bca72db483b4 (127.25.236.130:36681): tablet f82fe438aaa64365b8a36d18c0ae1646 (table test-table [id=f764c1c0068d4a38ba374ef67ba11c6a]) successfully deleted
I20260806 01:13:50.498155 30057 ts_tablet_manager.cc:1939] T f82fe438aaa64365b8a36d18c0ae1646 P c409779bcd184007bb77e07b0881fb20: Deleting consensus metadata
I20260806 01:13:50.499524 29121 catalog_manager.cc:4841] TS c409779bcd184007bb77e07b0881fb20 (127.25.236.129:35167): tablet f82fe438aaa64365b8a36d18c0ae1646 (table test-table [id=f764c1c0068d4a38ba374ef67ba11c6a]) successfully deleted
Aug 06 01:13:50 dist-test-slave-7nm7 krb5kdc[29076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:13:50 dist-test-slave-7nm7 krb5kdc[29076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:50.547487 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:50.526787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41034 (local address 127.25.236.190:37343)
0806 01:13:50.527424 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:50.527429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:50.527452 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:13:50.527559 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:50.527569 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:50.527629 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:50.527728 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:50.529018 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:50.530055 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:50.531874 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:50.532443 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:50.532544 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:50.540640 (+ 8096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:50.540645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:50.540649 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:50.540686 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:50.544149 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:50.544902 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:50.544906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:50.544909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:50.545185 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:50.545579 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:50.545583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:50.545585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:50.546007 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:50.546333 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:50.546926 (+ 593us) server_negotiation.cc:299] Negotiation successful
0806 01:13:50.547134 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":377,"threads_started":1}
I20260806 01:13:50.551088 29147 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:13:50.551796 29147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:13:50.558835 29353 tablet_service.cc:1467] Processing CreateTablet for tablet 89c5b76e0e0b4d3984790883e374e251 (DEFAULT_TABLE table=test-table [id=d6dcd3bde3ff406eae67517c3181f36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:50.559090 29562 tablet_service.cc:1467] Processing CreateTablet for tablet 89c5b76e0e0b4d3984790883e374e251 (DEFAULT_TABLE table=test-table [id=d6dcd3bde3ff406eae67517c3181f36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:50.559280 29353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89c5b76e0e0b4d3984790883e374e251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:50.559305 29759 tablet_service.cc:1467] Processing CreateTablet for tablet 89c5b76e0e0b4d3984790883e374e251 (DEFAULT_TABLE table=test-table [id=d6dcd3bde3ff406eae67517c3181f36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:50.559584 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89c5b76e0e0b4d3984790883e374e251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:50.559767 29759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89c5b76e0e0b4d3984790883e374e251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:50.574074 30076 tablet_bootstrap.cc:492] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20: Bootstrap starting.
I20260806 01:13:50.574986 30077 tablet_bootstrap.cc:492] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af: Bootstrap starting.
I20260806 01:13:50.574990 30078 tablet_bootstrap.cc:492] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4: Bootstrap starting.
I20260806 01:13:50.576637 30076 tablet_bootstrap.cc:654] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:50.577390 30077 tablet_bootstrap.cc:654] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:50.579324 30076 tablet_bootstrap.cc:492] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20: No bootstrap required, opened a new log
I20260806 01:13:50.579465 30076 ts_tablet_manager.cc:1397] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 01:13:50.580176 30077 tablet_bootstrap.cc:492] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af: No bootstrap required, opened a new log
I20260806 01:13:50.580363 30077 ts_tablet_manager.cc:1397] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260806 01:13:50.580377 30076 raft_consensus.cc:348] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.580646 30076 raft_consensus.cc:374] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:50.580749 30076 raft_consensus.cc:729] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c409779bcd184007bb77e07b0881fb20, State: Initialized, Role: FOLLOWER
I20260806 01:13:50.581041 30076 consensus_queue.cc:260] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.581451 30077 raft_consensus.cc:348] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.581802 30077 raft_consensus.cc:374] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:50.581916 30077 raft_consensus.cc:729] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec66fe008304da9a2e4a55e13a945af, State: Initialized, Role: FOLLOWER
I20260806 01:13:50.582334 30077 consensus_queue.cc:260] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.582903 30076 ts_tablet_manager.cc:1428] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 01:13:50.585431 30078 tablet_bootstrap.cc:654] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:50.586349 30077 ts_tablet_manager.cc:1428] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260806 01:13:50.588735 30078 tablet_bootstrap.cc:492] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4: No bootstrap required, opened a new log
I20260806 01:13:50.588894 30078 ts_tablet_manager.cc:1397] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.002s
I20260806 01:13:50.589731 30078 raft_consensus.cc:348] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.589991 30078 raft_consensus.cc:374] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:50.590099 30078 raft_consensus.cc:729] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048d166145f0424cbd94bca72db483b4, State: Initialized, Role: FOLLOWER
I20260806 01:13:50.590389 30078 consensus_queue.cc:260] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.590888 30078 ts_tablet_manager.cc:1428] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 01:13:50.768400 29916 raft_consensus.cc:482] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:13:50.768705 29916 raft_consensus.cc:504] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.769733 29916 leader_election.cc:288] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c409779bcd184007bb77e07b0881fb20 (127.25.236.129:35167), eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229)
I20260806 01:13:50.770367 29393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c5b76e0e0b4d3984790883e374e251" candidate_uuid: "048d166145f0424cbd94bca72db483b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c409779bcd184007bb77e07b0881fb20" is_pre_election: true
I20260806 01:13:50.770675 29393 raft_consensus.cc:2454] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 048d166145f0424cbd94bca72db483b4 in term 0.
I20260806 01:13:50.770517 29806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c5b76e0e0b4d3984790883e374e251" candidate_uuid: "048d166145f0424cbd94bca72db483b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec66fe008304da9a2e4a55e13a945af" is_pre_election: true
I20260806 01:13:50.770917 29806 raft_consensus.cc:2454] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 048d166145f0424cbd94bca72db483b4 in term 0.
I20260806 01:13:50.771245 29494 leader_election.cc:302] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048d166145f0424cbd94bca72db483b4, c409779bcd184007bb77e07b0881fb20; no voters:
I20260806 01:13:50.771663 29916 raft_consensus.cc:2781] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:13:50.771806 29916 raft_consensus.cc:482] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:13:50.771909 29916 raft_consensus.cc:3037] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:50.774071 29916 raft_consensus.cc:504] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.774803 29916 leader_election.cc:288] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [CANDIDATE]: Term 1 election: Requested vote from peers c409779bcd184007bb77e07b0881fb20 (127.25.236.129:35167), eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229)
I20260806 01:13:50.775280 29393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c5b76e0e0b4d3984790883e374e251" candidate_uuid: "048d166145f0424cbd94bca72db483b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c409779bcd184007bb77e07b0881fb20"
I20260806 01:13:50.775488 29393 raft_consensus.cc:3037] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:50.775421 29806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c5b76e0e0b4d3984790883e374e251" candidate_uuid: "048d166145f0424cbd94bca72db483b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec66fe008304da9a2e4a55e13a945af"
I20260806 01:13:50.775662 29806 raft_consensus.cc:3037] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:50.777750 29393 raft_consensus.cc:2454] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 048d166145f0424cbd94bca72db483b4 in term 1.
I20260806 01:13:50.777773 29806 raft_consensus.cc:2454] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 048d166145f0424cbd94bca72db483b4 in term 1.
I20260806 01:13:50.778270 29494 leader_election.cc:302] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048d166145f0424cbd94bca72db483b4, c409779bcd184007bb77e07b0881fb20; no voters:
I20260806 01:13:50.778609 29916 raft_consensus.cc:2781] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:50.778787 29916 raft_consensus.cc:686] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [term 1 LEADER]: Becoming Leader. State: Replica: 048d166145f0424cbd94bca72db483b4, State: Running, Role: LEADER
I20260806 01:13:50.779116 29916 consensus_queue.cc:237] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.782051 29146 catalog_manager.cc:5495] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 048d166145f0424cbd94bca72db483b4 (127.25.236.130). New cstate: current_term: 1 leader_uuid: "048d166145f0424cbd94bca72db483b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } health_report { overall_health: UNKNOWN } } }
I20260806 01:13:50.830114 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:50.824132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47734 (local address 127.25.236.130:36681)
0806 01:13:50.824805 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:50.824809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:50.824828 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:13:50.824879 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:50.824882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:50.824944 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:50.825057 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:50.825485 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:50.826262 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:50.827935 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:50.828422 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:50.828584 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:50.829127 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:50.829258 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:50.829373 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:50.829552 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 01:13:50.829638 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":332,"threads_started":1}
I20260806 01:13:50.848534 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:50.842827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:48129 (local address 127.25.236.130:36681)
0806 01:13:50.843111 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:50.843116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:50.843438 (+ 322us) server_negotiation.cc:407] Connection header received
0806 01:13:50.843585 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:50.843589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:50.843660 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:50.843737 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:50.844162 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:50.844939 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:50.846809 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:50.847323 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:50.847422 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:50.847880 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:50.847978 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:50.848047 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:50.848243 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 01:13:50.848328 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:13:50.848677 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:50.842373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:36681 (local address 127.25.236.130:48129)
0806 01:13:50.843340 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:50.843356 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:50.843484 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:50.843769 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:50.843772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:50.843780 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:13:50.844009 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:50.844016 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:50.845073 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:50.845077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:50.846655 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:50.846674 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:50.847495 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:50.847500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:50.847647 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:50.847697 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:13:50.848067 (+ 370us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:13:50.848076 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 01:13:50.848158 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 01:13:50.848262 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":667,"threads_started":1}
I20260806 01:13:50.858560 29393 raft_consensus.cc:1261] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Refusing update from remote peer 048d166145f0424cbd94bca72db483b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:50.858876 29806 raft_consensus.cc:1261] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Refusing update from remote peer 048d166145f0424cbd94bca72db483b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:50.859228 30099 consensus_queue.cc:1035] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:50.859839 30029 consensus_queue.cc:1035] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:50.912478 29146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:13:50.912762 29146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:13:50.915786 29146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d: Sending DeleteTablet for 3 replicas of tablet 89c5b76e0e0b4d3984790883e374e251
I20260806 01:13:50.916545 29353 tablet_service.cc:1514] Processing DeleteTablet for tablet 89c5b76e0e0b4d3984790883e374e251 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:50 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:54026
I20260806 01:13:50.916831 30057 tablet_replica.cc:330] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20: stopping tablet replica
I20260806 01:13:50.916942 29560 tablet_service.cc:1514] Processing DeleteTablet for tablet 89c5b76e0e0b4d3984790883e374e251 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:50 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:47712
I20260806 01:13:50.916976 29759 tablet_service.cc:1514] Processing DeleteTablet for tablet 89c5b76e0e0b4d3984790883e374e251 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:50 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:57532
I20260806 01:13:50.917066 30057 raft_consensus.cc:2229] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:13:50.917255 30057 raft_consensus.cc:2258] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:50.917255 30060 tablet_replica.cc:330] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af: stopping tablet replica
I20260806 01:13:50.917287 30058 tablet_replica.cc:330] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4: stopping tablet replica
I20260806 01:13:50.917452 30060 raft_consensus.cc:2229] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:13:50.917474 30058 raft_consensus.cc:2229] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:13:50.917654 30060 raft_consensus.cc:2258] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:50.917999 30058 raft_consensus.cc:2258] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:50.918723 30057 ts_tablet_manager.cc:1905] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:50.919082 30060 ts_tablet_manager.cc:1905] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:50.918915 29146 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:13:50.919715 29146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:13:50.920336 30058 ts_tablet_manager.cc:1905] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:50.926713 30060 ts_tablet_manager.cc:1918] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:13:50.927568 30058 ts_tablet_manager.cc:1918] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:13:50.927759 30058 log.cc:1196] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/89c5b76e0e0b4d3984790883e374e251
I20260806 01:13:50.928228 30058 ts_tablet_manager.cc:1939] T 89c5b76e0e0b4d3984790883e374e251 P 048d166145f0424cbd94bca72db483b4: Deleting consensus metadata
I20260806 01:13:50.929087 30057 ts_tablet_manager.cc:1918] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:13:50.929256 30057 log.cc:1196] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/89c5b76e0e0b4d3984790883e374e251
I20260806 01:13:50.929657 30057 ts_tablet_manager.cc:1939] T 89c5b76e0e0b4d3984790883e374e251 P c409779bcd184007bb77e07b0881fb20: Deleting consensus metadata
I20260806 01:13:50.929800 29122 catalog_manager.cc:4841] TS 048d166145f0424cbd94bca72db483b4 (127.25.236.130:36681): tablet 89c5b76e0e0b4d3984790883e374e251 (table test-table [id=d6dcd3bde3ff406eae67517c3181f36b]) successfully deleted
I20260806 01:13:50.930783 29121 catalog_manager.cc:4841] TS c409779bcd184007bb77e07b0881fb20 (127.25.236.129:35167): tablet 89c5b76e0e0b4d3984790883e374e251 (table test-table [id=d6dcd3bde3ff406eae67517c3181f36b]) successfully deleted
I20260806 01:13:50.931483 30060 log.cc:1196] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/89c5b76e0e0b4d3984790883e374e251
I20260806 01:13:50.931617 29353 tablet_service.cc:1467] Processing CreateTablet for tablet 3eaf069ecc2942388ec0ebe380567ade (DEFAULT_TABLE table=test-table [id=37d3f0d2d074482d8ebc237af32b962f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:50.932060 29353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eaf069ecc2942388ec0ebe380567ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:50.932078 30060 ts_tablet_manager.cc:1939] T 89c5b76e0e0b4d3984790883e374e251 P eec66fe008304da9a2e4a55e13a945af: Deleting consensus metadata
I20260806 01:13:50.932519 29759 tablet_service.cc:1467] Processing CreateTablet for tablet 3eaf069ecc2942388ec0ebe380567ade (DEFAULT_TABLE table=test-table [id=37d3f0d2d074482d8ebc237af32b962f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:50.932988 29759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eaf069ecc2942388ec0ebe380567ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:50.934206 29560 tablet_service.cc:1467] Processing CreateTablet for tablet 3eaf069ecc2942388ec0ebe380567ade (DEFAULT_TABLE table=test-table [id=37d3f0d2d074482d8ebc237af32b962f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:13:50.934926 29560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eaf069ecc2942388ec0ebe380567ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:50.938731 29122 catalog_manager.cc:4841] TS eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229): tablet 89c5b76e0e0b4d3984790883e374e251 (table test-table [id=d6dcd3bde3ff406eae67517c3181f36b]) successfully deleted
I20260806 01:13:50.940665 30076 tablet_bootstrap.cc:492] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20: Bootstrap starting.
I20260806 01:13:50.940912 30077 tablet_bootstrap.cc:492] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af: Bootstrap starting.
I20260806 01:13:50.942646 30076 tablet_bootstrap.cc:654] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:50.942936 30077 tablet_bootstrap.cc:654] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:50.946707 30077 tablet_bootstrap.cc:492] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af: No bootstrap required, opened a new log
I20260806 01:13:50.946906 30077 ts_tablet_manager.cc:1397] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 01:13:50.947319 30076 tablet_bootstrap.cc:492] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20: No bootstrap required, opened a new log
I20260806 01:13:50.947561 30076 ts_tablet_manager.cc:1397] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 01:13:50.948364 30076 raft_consensus.cc:348] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.948526 30078 tablet_bootstrap.cc:492] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4: Bootstrap starting.
I20260806 01:13:50.948716 30076 raft_consensus.cc:374] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:50.948616 30077 raft_consensus.cc:348] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.948905 30077 raft_consensus.cc:374] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:50.948904 30076 raft_consensus.cc:729] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c409779bcd184007bb77e07b0881fb20, State: Initialized, Role: FOLLOWER
I20260806 01:13:50.948995 30077 raft_consensus.cc:729] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec66fe008304da9a2e4a55e13a945af, State: Initialized, Role: FOLLOWER
I20260806 01:13:50.949363 30076 consensus_queue.cc:260] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.949424 30077 consensus_queue.cc:260] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.950002 30077 ts_tablet_manager.cc:1428] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 01:13:50.950502 30078 tablet_bootstrap.cc:654] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:50.951555 30076 ts_tablet_manager.cc:1428] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 01:13:50.959621 30078 tablet_bootstrap.cc:492] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4: No bootstrap required, opened a new log
I20260806 01:13:50.959841 30078 ts_tablet_manager.cc:1397] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20260806 01:13:50.960570 30078 raft_consensus.cc:348] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.960922 30078 raft_consensus.cc:374] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:50.961069 30078 raft_consensus.cc:729] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048d166145f0424cbd94bca72db483b4, State: Initialized, Role: FOLLOWER
I20260806 01:13:50.961442 30078 consensus_queue.cc:260] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:50.962118 30078 ts_tablet_manager.cc:1428] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260806 01:13:51.028936 30099 raft_consensus.cc:482] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:13:51.029263 30099 raft_consensus.cc:504] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:51.030138 30099 leader_election.cc:288] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c409779bcd184007bb77e07b0881fb20 (127.25.236.129:35167), eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229)
I20260806 01:13:51.030773 29805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eaf069ecc2942388ec0ebe380567ade" candidate_uuid: "048d166145f0424cbd94bca72db483b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec66fe008304da9a2e4a55e13a945af" is_pre_election: true
I20260806 01:13:51.030848 29396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eaf069ecc2942388ec0ebe380567ade" candidate_uuid: "048d166145f0424cbd94bca72db483b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c409779bcd184007bb77e07b0881fb20" is_pre_election: true
I20260806 01:13:51.031112 29396 raft_consensus.cc:2454] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 048d166145f0424cbd94bca72db483b4 in term 0.
I20260806 01:13:51.031101 29805 raft_consensus.cc:2454] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 048d166145f0424cbd94bca72db483b4 in term 0.
I20260806 01:13:51.031580 29494 leader_election.cc:302] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048d166145f0424cbd94bca72db483b4, c409779bcd184007bb77e07b0881fb20; no voters:
I20260806 01:13:51.031985 30099 raft_consensus.cc:2781] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:13:51.032099 30099 raft_consensus.cc:482] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:13:51.032379 30099 raft_consensus.cc:3037] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:51.035440 30099 raft_consensus.cc:504] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:51.036137 30099 leader_election.cc:288] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [CANDIDATE]: Term 1 election: Requested vote from peers c409779bcd184007bb77e07b0881fb20 (127.25.236.129:35167), eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229)
I20260806 01:13:51.036631 29396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eaf069ecc2942388ec0ebe380567ade" candidate_uuid: "048d166145f0424cbd94bca72db483b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c409779bcd184007bb77e07b0881fb20"
I20260806 01:13:51.036680 29805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eaf069ecc2942388ec0ebe380567ade" candidate_uuid: "048d166145f0424cbd94bca72db483b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec66fe008304da9a2e4a55e13a945af"
I20260806 01:13:51.036931 29805 raft_consensus.cc:3037] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:51.036940 29396 raft_consensus.cc:3037] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:51.040162 29396 raft_consensus.cc:2454] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 048d166145f0424cbd94bca72db483b4 in term 1.
I20260806 01:13:51.040681 29494 leader_election.cc:302] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048d166145f0424cbd94bca72db483b4, c409779bcd184007bb77e07b0881fb20; no voters:
I20260806 01:13:51.041033 30099 raft_consensus.cc:2781] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:51.041240 30099 raft_consensus.cc:686] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [term 1 LEADER]: Becoming Leader. State: Replica: 048d166145f0424cbd94bca72db483b4, State: Running, Role: LEADER
I20260806 01:13:51.041527 30099 consensus_queue.cc:237] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } }
I20260806 01:13:51.042704 29805 raft_consensus.cc:2454] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 048d166145f0424cbd94bca72db483b4 in term 1.
I20260806 01:13:51.045403 29145 catalog_manager.cc:5495] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 048d166145f0424cbd94bca72db483b4 (127.25.236.130). New cstate: current_term: 1 leader_uuid: "048d166145f0424cbd94bca72db483b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "048d166145f0424cbd94bca72db483b4" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 } health_report { overall_health: UNKNOWN } } }
I20260806 01:13:51.094787 29396 raft_consensus.cc:1261] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Refusing update from remote peer 048d166145f0424cbd94bca72db483b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:51.095053 29806 raft_consensus.cc:1261] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Refusing update from remote peer 048d166145f0424cbd94bca72db483b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:51.095772 30099 consensus_queue.cc:1035] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eec66fe008304da9a2e4a55e13a945af" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 33229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:51.096405 30099 consensus_queue.cc:1035] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c409779bcd184007bb77e07b0881fb20" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:13:51.123610 30027 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 01:13:52.726898 29146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:13:52.727123 29146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:13:52.732143 29146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7fb91a3ee2f74327bd283642b185fe3d: Sending DeleteTablet for 3 replicas of tablet 3eaf069ecc2942388ec0ebe380567ade
I20260806 01:13:52.733373 29353 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eaf069ecc2942388ec0ebe380567ade with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:52 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:54026
I20260806 01:13:52.733788 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 29247
I20260806 01:13:52.733959 29759 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eaf069ecc2942388ec0ebe380567ade with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:52 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:57532
I20260806 01:13:52.734566 29560 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eaf069ecc2942388ec0ebe380567ade with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:13:52 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:47712
I20260806 01:13:52.744926 30142 tablet_replica.cc:330] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20: stopping tablet replica
I20260806 01:13:52.745386 30142 raft_consensus.cc:2229] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:13:52.745728 30142 raft_consensus.cc:2258] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:52.747098 30142 ts_tablet_manager.cc:1905] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:52.747561 30144 tablet_replica.cc:330] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4: stopping tablet replica
I20260806 01:13:52.747843 30144 raft_consensus.cc:2229] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:13:52.748322 30144 raft_consensus.cc:2258] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:52.749614 30144 ts_tablet_manager.cc:1905] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:13:52.757296 30142 ts_tablet_manager.cc:1918] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:13:52.757519 30142 log.cc:1196] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/3eaf069ecc2942388ec0ebe380567ade
I20260806 01:13:52.757972 30144 ts_tablet_manager.cc:1918] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:13:52.758131 30142 ts_tablet_manager.cc:1939] T 3eaf069ecc2942388ec0ebe380567ade P c409779bcd184007bb77e07b0881fb20: Deleting consensus metadata
I20260806 01:13:52.758160 30144 log.cc:1196] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/3eaf069ecc2942388ec0ebe380567ade
I20260806 01:13:52.758682 30144 ts_tablet_manager.cc:1939] T 3eaf069ecc2942388ec0ebe380567ade P 048d166145f0424cbd94bca72db483b4: Deleting consensus metadata
I20260806 01:13:52.760138 29122 catalog_manager.cc:4841] TS 048d166145f0424cbd94bca72db483b4 (127.25.236.130:36681): tablet 3eaf069ecc2942388ec0ebe380567ade (table test-table [id=37d3f0d2d074482d8ebc237af32b962f]) successfully deleted
I20260806 01:13:52.760346 29121 catalog_manager.cc:4841] TS c409779bcd184007bb77e07b0881fb20 (127.25.236.129:35167): tablet 3eaf069ecc2942388ec0ebe380567ade (table test-table [id=37d3f0d2d074482d8ebc237af32b962f]) successfully deleted
I20260806 01:13:52.761128 30143 tablet_replica.cc:330] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af: stopping tablet replica
I20260806 01:13:52.761370 30143 raft_consensus.cc:2229] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:13:52.761658 30143 raft_consensus.cc:2258] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:13:52.762635 30143 ts_tablet_manager.cc:1905] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 01:13:52 dist-test-slave-7nm7 krb5kdc[29076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:13:52.769405 30143 ts_tablet_manager.cc:1918] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:13:52.769558 30143 log.cc:1196] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/3eaf069ecc2942388ec0ebe380567ade
I20260806 01:13:52.769989 30143 ts_tablet_manager.cc:1939] T 3eaf069ecc2942388ec0ebe380567ade P eec66fe008304da9a2e4a55e13a945af: Deleting consensus metadata
I20260806 01:13:52.771396 29122 catalog_manager.cc:4841] TS eec66fe008304da9a2e4a55e13a945af (127.25.236.131:33229): tablet 3eaf069ecc2942388ec0ebe380567ade (table test-table [id=37d3f0d2d074482d8ebc237af32b962f]) successfully deleted
I20260806 01:13:52.778344 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:52.736310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54032 (local address 127.25.236.129:35167)
0806 01:13:52.759547 (+ 23237us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:52.759553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:52.759579 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:13:52.759637 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:52.759643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:52.759736 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:52.759880 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:52.761629 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:52.762646 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:52.764436 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:52.764885 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:52.764996 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:52.770356 (+ 5360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:52.770376 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:52.770398 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:52.770442 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:52.773999 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:52.775702 (+ 1703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:52.775713 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:52.775715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:52.775885 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:52.776436 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:52.776442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:52.776445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:52.777163 (+ 718us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:52.777474 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:52.777767 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 01:13:52.777952 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23081,"thread_start_us":23731,"threads_started":1}
W20260806 01:13:52.779058 29457 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54032
W20260806 01:13:52.779940 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 29247: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:13:52.780434 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 29247
I20260806 01:13:52.819237 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 29468
Aug 06 01:13:52 dist-test-slave-7nm7 krb5kdc[29076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:13:52.837808 30147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:52.820206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47748 (local address 127.25.236.130:36681)
0806 01:13:52.820830 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:52.820836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:52.820854 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:13:52.820926 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:52.820930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:52.821001 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:52.821116 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:52.822668 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:52.823525 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:52.825212 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:52.825637 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:52.825742 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:52.830395 (+ 4653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:52.830411 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:52.830428 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:52.830465 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:52.834805 (+ 4340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:52.835616 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:52.835625 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:52.835627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:52.835767 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:52.836170 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:52.836175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:52.836177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:52.836547 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:52.836883 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:52.837186 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 01:13:52.837423 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":322,"threads_started":1}
W20260806 01:13:52.838901 29658 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47748
W20260806 01:13:52.839794 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 29468: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:13:52.839927 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 29468
I20260806 01:13:52.887630 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 29672
Aug 06 01:13:52 dist-test-slave-7nm7 krb5kdc[29076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:13:52.906931 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:52.888451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.25.236.131:33229)
0806 01:13:52.889040 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:52.889045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:52.889062 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:13:52.889246 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:52.889256 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:52.889403 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:52.889622 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:52.891119 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:52.891931 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:52.893456 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:52.894049 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:52.894211 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:52.899688 (+ 5477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:52.899709 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:52.899728 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:52.899774 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:52.903652 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:52.904484 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:52.904498 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:52.904502 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:52.904819 (+ 317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:52.905283 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:52.905288 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:52.905290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:52.905720 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:52.906000 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:52.906426 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 01:13:52.906640 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":282,"threads_started":1}
W20260806 01:13:52.907853 29871 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57562
W20260806 01:13:52.908527 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 29672: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:13:52.908689 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 29672
I20260806 01:13:52.945842 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 29094
I20260806 01:13:52.947077 29202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:53.072737 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 29094
2026-08-06T01:13:53Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9859 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:13:53 dist-test-slave-7nm7 krb5kdc[30153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:13:53 dist-test-slave-7nm7 krb5kdc[30153](info): set up 2 sockets
Aug 06 01:13:53 dist-test-slave-7nm7 krb5kdc[30153](info): commencing operation
krb5kdc: starting...
W20260806 01:13:54.444340 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:13:54 dist-test-slave-7nm7 krb5kdc[30153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:13:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:13:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:13:54.700150 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:42521
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:40333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:42521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:55.061985 30175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:55.062379 30175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:55.062556 30175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:55.070318 30175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:55.070478 30175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:55.070596 30175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:55.070698 30175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:55.070785 30175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:55.081084 30175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:40333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:42521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:42521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:55.082566 30175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:55.083961 30175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:55.096453 30181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:55.096519 30184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:55.096441 30182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:55.100325 30175 server_base.cc:1034] running on GCE node
I20260806 01:13:55.101346 30175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:55.102931 30175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:55.104205 30175 hybrid_clock.cc:648] HybridClock initialized: now 1785978835104139 us; error 58 us; skew 500 ppm
I20260806 01:13:55.104679 30175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:55 dist-test-slave-7nm7 krb5kdc[30153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978835, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:55.108083 30175 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:13:55.109841 30175 webserver.cc:460] Webserver started at http://127.25.236.190:44021/ using document root <none> and password file <none>
I20260806 01:13:55.110688 30175 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:55.110845 30175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:55.111527 30175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:55.116807 30175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "9b0027c0341c4b19aa048a7f10c1e24d"
format_stamp: "Formatted at 2026-08-06 01:13:55 on dist-test-slave-7nm7"
server_key: "d369a0629dc5e30680ef6098c16bd442"
server_key_iv: "42d1a5d55a65799c09db960e82fdcc1e"
server_key_version: "encryptionkey@0"
I20260806 01:13:55.117895 30175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "9b0027c0341c4b19aa048a7f10c1e24d"
format_stamp: "Formatted at 2026-08-06 01:13:55 on dist-test-slave-7nm7"
server_key: "d369a0629dc5e30680ef6098c16bd442"
server_key_iv: "42d1a5d55a65799c09db960e82fdcc1e"
server_key_version: "encryptionkey@0"
I20260806 01:13:55.124347 30175 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 01:13:55.128866 30198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:55.130707 30175 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 01:13:55.130944 30175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "9b0027c0341c4b19aa048a7f10c1e24d"
format_stamp: "Formatted at 2026-08-06 01:13:55 on dist-test-slave-7nm7"
server_key: "d369a0629dc5e30680ef6098c16bd442"
server_key_iv: "42d1a5d55a65799c09db960e82fdcc1e"
server_key_version: "encryptionkey@0"
I20260806 01:13:55.131177 30175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:55.145180 30175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:55.150014 30175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:55.150449 30175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:55.174454 30175 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:42521
I20260806 01:13:55.174382 30273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:42521 every 8 connection(s)
I20260806 01:13:55.176626 30175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:13:55.177106 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 30175
I20260806 01:13:55.177384 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:13:55.178448 26546 external_mini_cluster.cc:1428] Setting key f9438a48b7efc92caac54ab2eb41fe68
I20260806 01:13:55.182325 30274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:13:55 dist-test-slave-7nm7 krb5kdc[30153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:55.194674 30274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d: Bootstrap starting.
I20260806 01:13:55.199146 30274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:55.200783 30274 log.cc:824] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:55.204100 30274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d: No bootstrap required, opened a new log
I20260806 01:13:55.204675 30277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:55.181207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47862 (local address 127.25.236.190:42521)
0806 01:13:55.182053 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:55.182066 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:55.182111 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:13:55.184045 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:55.184068 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:55.184452 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:55.184918 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:55.186789 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:55.188722 (+ 1933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:55.190227 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:55.190960 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:55.191182 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:55.195686 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:55.195698 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:55.195718 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:55.195771 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:55.198976 (+ 3205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:55.199802 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:55.199809 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:55.199815 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:55.200057 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:55.200485 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:55.200500 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:55.200503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:55.201225 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:55.201737 (+ 512us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:55.202269 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 01:13:55.202710 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":394,"threads_started":1}
I20260806 01:13:55.212615 30274 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0027c0341c4b19aa048a7f10c1e24d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 42521 } }
I20260806 01:13:55.213034 30274 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:55.213151 30274 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b0027c0341c4b19aa048a7f10c1e24d, State: Initialized, Role: FOLLOWER
I20260806 01:13:55.213889 30274 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0027c0341c4b19aa048a7f10c1e24d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 42521 } }
I20260806 01:13:55.214195 30274 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:13:55.214375 30274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:13:55.214576 30274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:55.216984 30274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0027c0341c4b19aa048a7f10c1e24d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 42521 } }
I20260806 01:13:55.217634 30274 leader_election.cc:302] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b0027c0341c4b19aa048a7f10c1e24d; no voters:
I20260806 01:13:55.218703 30274 leader_election.cc:288] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:13:55.218811 30279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:55.219621 30279 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [term 1 LEADER]: Becoming Leader. State: Replica: 9b0027c0341c4b19aa048a7f10c1e24d, State: Running, Role: LEADER
I20260806 01:13:55.220343 30279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0027c0341c4b19aa048a7f10c1e24d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 42521 } }
I20260806 01:13:55.221557 30274 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:55.224501 30283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b0027c0341c4b19aa048a7f10c1e24d. Latest consensus state: current_term: 1 leader_uuid: "9b0027c0341c4b19aa048a7f10c1e24d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0027c0341c4b19aa048a7f10c1e24d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 42521 } } }
I20260806 01:13:55.225025 30283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:55.224476 30280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b0027c0341c4b19aa048a7f10c1e24d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0027c0341c4b19aa048a7f10c1e24d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 42521 } } }
I20260806 01:13:55.225204 30280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:55.230796 30291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:13:55.237763 30291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:13:55.249614 30291 catalog_manager.cc:1347] Generated new cluster ID: 284ed02d924a4cb5a23cc0135540ecc8
I20260806 01:13:55.249753 30291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:13:55.268946 30291 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:13:55.270924 30291 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:13:55.278828 30291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b0027c0341c4b19aa048a7f10c1e24d: Generated new TSK 0
I20260806 01:13:55.279721 30291 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:13:55.349782 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:42521
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:40333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:55.732252 30313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:55.732609 30313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:55.732770 30313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:55.741014 30313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:55.741160 30313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:55.741391 30313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:13:55.753500 30313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:40333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:42521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:55.755010 30313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:55.756369 30313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:55.770498 30318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:55.770395 30324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:55.770668 30320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:55.771976 30313 server_base.cc:1034] running on GCE node
I20260806 01:13:55.772754 30313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:55.773811 30313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:55.775055 30313 hybrid_clock.cc:648] HybridClock initialized: now 1785978835775000 us; error 48 us; skew 500 ppm
I20260806 01:13:55.775638 30313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:55 dist-test-slave-7nm7 krb5kdc[30153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978835, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:55.783942 30313 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:13:55.786746 30313 webserver.cc:460] Webserver started at http://127.25.236.129:38183/ using document root <none> and password file <none>
I20260806 01:13:55.787717 30313 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:55.787897 30313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:55.788266 30313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:55.791527 30313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "5e5cce03f4894ff88814a00fcaa430de"
format_stamp: "Formatted at 2026-08-06 01:13:55 on dist-test-slave-7nm7"
server_key: "232002a6d18e15bc4fde110064eb5d49"
server_key_iv: "7afc1cc00dedfe3297ab4cbef52801af"
server_key_version: "encryptionkey@0"
I20260806 01:13:55.792398 30313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "5e5cce03f4894ff88814a00fcaa430de"
format_stamp: "Formatted at 2026-08-06 01:13:55 on dist-test-slave-7nm7"
server_key: "232002a6d18e15bc4fde110064eb5d49"
server_key_iv: "7afc1cc00dedfe3297ab4cbef52801af"
server_key_version: "encryptionkey@0"
I20260806 01:13:55.799090 30313 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260806 01:13:55.803062 30336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:55.804638 30313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:13:55.804884 30313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "5e5cce03f4894ff88814a00fcaa430de"
format_stamp: "Formatted at 2026-08-06 01:13:55 on dist-test-slave-7nm7"
server_key: "232002a6d18e15bc4fde110064eb5d49"
server_key_iv: "7afc1cc00dedfe3297ab4cbef52801af"
server_key_version: "encryptionkey@0"
I20260806 01:13:55.805198 30313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:55.823526 30313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:55.828034 30313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:55.828367 30313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:55.829530 30313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:13:55.831048 30313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:55.831152 30313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:55.831343 30313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:55.831458 30313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:55.872334 30313 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:41777
I20260806 01:13:55.872565 30510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:41777 every 8 connection(s)
I20260806 01:13:55.873728 30313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:13:55.877228 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 30313
I20260806 01:13:55.877480 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:13:55.878211 26546 external_mini_cluster.cc:1428] Setting key 090a288cfba43f9665f43b2a4ec17763
Aug 06 01:13:55 dist-test-slave-7nm7 krb5kdc[30153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978835, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:55.897841 30518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:55.876474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:39563 (local address 127.25.236.190:42521)
0806 01:13:55.877078 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:55.877092 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:55.878064 (+ 972us) server_negotiation.cc:407] Connection header received
0806 01:13:55.880072 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:55.880077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:55.880175 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:55.880254 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:55.881913 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:55.882725 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:55.884395 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:55.884833 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:55.884976 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:55.889379 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:55.889389 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:55.889397 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:55.889451 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:55.893549 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:55.894819 (+ 1270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:55.894825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:55.894827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:55.895006 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:55.895583 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:55.895587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:55.895589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:55.895901 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:55.896025 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:55.897232 (+ 1207us) server_negotiation.cc:299] Negotiation successful
0806 01:13:55.897559 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":298,"threads_started":1}
I20260806 01:13:55.900509 30520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:55.877021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:42521 (local address 127.25.236.129:39563)
0806 01:13:55.877868 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:55.877924 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:55.879827 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:55.880658 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:55.880667 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:55.881171 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:55.881733 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:55.881750 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:55.882892 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:55.882969 (+ 77us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:55.884221 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:55.884235 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:55.885091 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:55.885099 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:55.885342 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:55.886356 (+ 1014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:55.886383 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:55.888994 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:55.893791 (+ 4797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:55.893808 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:55.893833 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:55.894610 (+ 777us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:55.895139 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:55.895148 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:55.895154 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:55.895434 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:55.896053 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:55.896065 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:55.896720 (+ 655us) client_negotiation.cc:769] Sending connection context
0806 01:13:55.897063 (+ 343us) client_negotiation.cc:240] Negotiation successful
0806 01:13:55.897846 (+ 783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":359,"threads_started":1}
I20260806 01:13:55.902606 30511 heartbeater.cc:347] Connected to a master server at 127.25.236.190:42521
I20260806 01:13:55.902946 30511 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:55.903751 30511 heartbeater.cc:511] Master 127.25.236.190:42521 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
I20260806 01:13:55.906870 30220 ts_manager.cc:194] Registered new tserver with Master: 5e5cce03f4894ff88814a00fcaa430de (127.25.236.129:41777)
I20260806 01:13:55.908551 30220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:39563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:13:55.937031 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:42521
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:40333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:56.307108 30526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:56.307463 30526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:56.307624 30526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:56.315145 30526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:56.315392 30526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:56.315637 30526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:13:56.326649 30526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:40333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:42521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:56.328161 30526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:56.329866 30526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:56.344650 30534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:56.344878 30531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:56.344853 30532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:56.345669 30526 server_base.cc:1034] running on GCE node
I20260806 01:13:56.346472 30526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:56.347683 30526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:56.348950 30526 hybrid_clock.cc:648] HybridClock initialized: now 1785978836348912 us; error 74 us; skew 500 ppm
I20260806 01:13:56.349321 30526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:56 dist-test-slave-7nm7 krb5kdc[30153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978836, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:56.352713 30526 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:13:56.354391 30526 webserver.cc:460] Webserver started at http://127.25.236.130:34831/ using document root <none> and password file <none>
I20260806 01:13:56.355134 30526 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:56.355293 30526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:56.355610 30526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:56.358656 30526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "a4b91dc7f85b4c2dbd0bdca8cceecb99"
format_stamp: "Formatted at 2026-08-06 01:13:56 on dist-test-slave-7nm7"
server_key: "5f4c553f66fd1a7b10214d36498dfc24"
server_key_iv: "20605aaf22d63bb8bb13cd51f80d101b"
server_key_version: "encryptionkey@0"
I20260806 01:13:56.359465 30526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "a4b91dc7f85b4c2dbd0bdca8cceecb99"
format_stamp: "Formatted at 2026-08-06 01:13:56 on dist-test-slave-7nm7"
server_key: "5f4c553f66fd1a7b10214d36498dfc24"
server_key_iv: "20605aaf22d63bb8bb13cd51f80d101b"
server_key_version: "encryptionkey@0"
I20260806 01:13:56.364864 30526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:13:56.368517 30542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:56.369941 30526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:13:56.370149 30526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "a4b91dc7f85b4c2dbd0bdca8cceecb99"
format_stamp: "Formatted at 2026-08-06 01:13:56 on dist-test-slave-7nm7"
server_key: "5f4c553f66fd1a7b10214d36498dfc24"
server_key_iv: "20605aaf22d63bb8bb13cd51f80d101b"
server_key_version: "encryptionkey@0"
I20260806 01:13:56.370522 30526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:56.391729 30526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:56.396518 30526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:56.396864 30526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:56.397912 30526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:13:56.399592 30526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:56.399703 30526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:56.399874 30526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:56.399955 30526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:56.446195 30526 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:36999
I20260806 01:13:56.446300 30705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:36999 every 8 connection(s)
I20260806 01:13:56.447947 30526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:13:56.452581 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 30526
I20260806 01:13:56.452937 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:13:56.453621 26546 external_mini_cluster.cc:1428] Setting key 75667f154cd730513a0b671c63a7d60e
Aug 06 01:13:56 dist-test-slave-7nm7 krb5kdc[30153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978836, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:56.471665 30709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:56.451303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:44997 (local address 127.25.236.190:42521)
0806 01:13:56.451782 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:56.451789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:56.452852 (+ 1063us) server_negotiation.cc:407] Connection header received
0806 01:13:56.454949 (+ 2097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:56.454954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:56.455050 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:56.455189 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:56.456686 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:56.457537 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:56.459903 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:56.460427 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:56.460608 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:56.464923 (+ 4315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:56.464929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:56.464932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:56.464966 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:56.468002 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:56.469028 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:56.469032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:56.469035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:56.469171 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:56.469718 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:56.469723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:56.469725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:56.470016 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:56.470318 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:56.471213 (+ 895us) server_negotiation.cc:299] Negotiation successful
0806 01:13:56.471387 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":346,"thread_start_us":216,"threads_started":1}
I20260806 01:13:56.473237 30710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:56.451825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:42521 (local address 127.25.236.130:44997)
0806 01:13:56.452675 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:56.452733 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:56.454682 (+ 1949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:56.455429 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:56.455437 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:56.455947 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:56.456513 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:56.456531 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:56.457724 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:56.457736 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:56.459536 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:56.459566 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:56.460751 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:56.460761 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:56.461021 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:56.461756 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:56.461794 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:56.464618 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:56.468178 (+ 3560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:56.468192 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:56.468213 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:56.468889 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:56.469285 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:56.469292 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:56.469298 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:56.469559 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:56.470276 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:56.470287 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:56.470774 (+ 487us) client_negotiation.cc:769] Sending connection context
0806 01:13:56.471094 (+ 320us) client_negotiation.cc:240] Negotiation successful
0806 01:13:56.471707 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":349,"threads_started":1}
I20260806 01:13:56.474923 30706 heartbeater.cc:347] Connected to a master server at 127.25.236.190:42521
I20260806 01:13:56.475288 30706 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:56.476032 30706 heartbeater.cc:511] Master 127.25.236.190:42521 requested a full tablet report, sending...
I20260806 01:13:56.477504 30220 ts_manager.cc:194] Registered new tserver with Master: a4b91dc7f85b4c2dbd0bdca8cceecb99 (127.25.236.130:36999)
I20260806 01:13:56.478411 30220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:44997
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:13:56.510707 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:42521
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:40333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:56.890551 30714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:56.890883 30714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:56.891058 30714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:56.898882 30714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:56.899040 30714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:56.899217 30714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:13:56.909204 30714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:40333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:42521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:56.910770 30714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:56.912149 30714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 01:13:56.912375 30511 heartbeater.cc:503] Master 127.25.236.190:42521 was elected leader, sending a full tablet report...
W20260806 01:13:56.925532 30722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:56.925488 30727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:56.925573 30723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:56.926759 30714 server_base.cc:1034] running on GCE node
I20260806 01:13:56.927368 30714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:56.928390 30714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:56.929632 30714 hybrid_clock.cc:648] HybridClock initialized: now 1785978836929588 us; error 64 us; skew 500 ppm
I20260806 01:13:56.929970 30714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:56 dist-test-slave-7nm7 krb5kdc[30153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978836, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:56.933661 30714 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:13:56.935398 30714 webserver.cc:460] Webserver started at http://127.25.236.131:34907/ using document root <none> and password file <none>
I20260806 01:13:56.936116 30714 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:56.936264 30714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:56.936575 30714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:56.939568 30714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "2cb645130e8c4200a5c8c958c2c9a921"
format_stamp: "Formatted at 2026-08-06 01:13:56 on dist-test-slave-7nm7"
server_key: "2b7ebbc45a50dcc8828f50ddbe68c1d3"
server_key_iv: "18f687745c4df419895689f523d61dd1"
server_key_version: "encryptionkey@0"
I20260806 01:13:56.940372 30714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "2cb645130e8c4200a5c8c958c2c9a921"
format_stamp: "Formatted at 2026-08-06 01:13:56 on dist-test-slave-7nm7"
server_key: "2b7ebbc45a50dcc8828f50ddbe68c1d3"
server_key_iv: "18f687745c4df419895689f523d61dd1"
server_key_version: "encryptionkey@0"
I20260806 01:13:56.945611 30714 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:13:56.949177 30737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:56.950754 30714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:13:56.950965 30714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "2cb645130e8c4200a5c8c958c2c9a921"
format_stamp: "Formatted at 2026-08-06 01:13:56 on dist-test-slave-7nm7"
server_key: "2b7ebbc45a50dcc8828f50ddbe68c1d3"
server_key_iv: "18f687745c4df419895689f523d61dd1"
server_key_version: "encryptionkey@0"
I20260806 01:13:56.951196 30714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:56.990420 30714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:56.995517 30714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:56.995857 30714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:56.996912 30714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:13:56.998916 30714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:13:56.999040 30714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:56.999366 30714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:13:56.999466 30714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:57.042706 30714 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:45133
I20260806 01:13:57.042739 30904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:45133 every 8 connection(s)
I20260806 01:13:57.044224 30714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:13:57.049383 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 30714
I20260806 01:13:57.049604 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:13:57.050184 26546 external_mini_cluster.cc:1428] Setting key 015491ee707af6e2a8a57af79442ebf9
Aug 06 01:13:57 dist-test-slave-7nm7 krb5kdc[30153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978836, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:57.068974 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:57.047424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:33225 (local address 127.25.236.190:42521)
0806 01:13:57.047976 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:57.047981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:57.049037 (+ 1056us) server_negotiation.cc:407] Connection header received
0806 01:13:57.051872 (+ 2835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:57.051879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:57.051975 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:57.052098 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:57.054279 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:57.055327 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:57.057224 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:57.057644 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:57.057762 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:57.061516 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:57.061521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:57.061525 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:57.061562 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:57.064337 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:57.065121 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:57.065125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:57.065127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:57.065251 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:57.065830 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:57.065837 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:57.065843 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:57.066668 (+ 825us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:57.066894 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:57.068311 (+ 1417us) server_negotiation.cc:299] Negotiation successful
0806 01:13:57.068567 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":303,"threads_started":1}
I20260806 01:13:57.069751 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:57.047939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:42521 (local address 127.25.236.131:33225)
0806 01:13:57.048791 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:57.048849 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:57.051496 (+ 2647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:57.052373 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:57.052386 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:57.053201 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:57.054069 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:57.054099 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:57.055549 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:57.055554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:57.057049 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:57.057064 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:57.057892 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:57.057897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:57.058131 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:57.058895 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:57.058919 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:57.061279 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:57.064477 (+ 3198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:57.064485 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:57.064498 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:57.064983 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:57.065395 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:57.065400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:57.065403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:57.065682 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:57.066976 (+ 1294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:57.066985 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:57.067620 (+ 635us) client_negotiation.cc:769] Sending connection context
0806 01:13:57.068026 (+ 406us) client_negotiation.cc:240] Negotiation successful
0806 01:13:57.068456 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":427,"threads_started":1}
I20260806 01:13:57.071136 30905 heartbeater.cc:347] Connected to a master server at 127.25.236.190:42521
I20260806 01:13:57.071476 30905 heartbeater.cc:464] Registering TS with master...
I20260806 01:13:57.072207 30905 heartbeater.cc:511] Master 127.25.236.190:42521 requested a full tablet report, sending...
I20260806 01:13:57.073689 30220 ts_manager.cc:194] Registered new tserver with Master: 2cb645130e8c4200a5c8c958c2c9a921 (127.25.236.131:45133)
I20260806 01:13:57.074745 30220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:33225
I20260806 01:13:57.081498 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 01:13:57.122524 30909 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:57.119541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47866 (local address 127.25.236.190:42521)
0806 01:13:57.119784 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:57.119791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:57.120285 (+ 494us) server_negotiation.cc:407] Connection header received
0806 01:13:57.120447 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:57.120451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:57.120522 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:57.120614 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:57.122327 (+ 1713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47866: BlockingRecv error: recv got EOF from 127.0.0.1:47866 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 01:13:57.126113 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 30313
W20260806 01:13:57.129323 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 30313: Not authorized: Client connection negotiation failed: client connection to 127.25.236.129:41777: server requires authentication, but client does not have Kerberos credentials available
I20260806 01:13:57.129530 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 30313
I20260806 01:13:57.154561 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 30526
W20260806 01:13:57.158694 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 30526: Not authorized: Client connection negotiation failed: client connection to 127.25.236.130:36999: server requires authentication, but client does not have Kerberos credentials available
I20260806 01:13:57.158883 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 30526
I20260806 01:13:57.186506 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 30714
W20260806 01:13:57.189703 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 30714: Not authorized: Client connection negotiation failed: client connection to 127.25.236.131:45133: server requires authentication, but client does not have Kerberos credentials available
I20260806 01:13:57.189853 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 30714
I20260806 01:13:57.214752 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 30175
I20260806 01:13:57.216557 30269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:57.328161 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 30175
2026-08-06T01:13:57Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4246 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:13:57 dist-test-slave-7nm7 krb5kdc[30934](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:13:57 dist-test-slave-7nm7 krb5kdc[30934](info): set up 2 sockets
Aug 06 01:13:57 dist-test-slave-7nm7 krb5kdc[30934](info): commencing operation
krb5kdc: starting...
W20260806 01:13:58.681582 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:13:58 dist-test-slave-7nm7 krb5kdc[30934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:13:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:13:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:13:58.952526 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:40555
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:46691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:40555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:59.324436 30957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:59.324884 30957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:59.325078 30957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:59.332805 30957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:59.332995 30957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:59.333262 30957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:59.333420 30957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:59.333519 30957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:59.344591 30957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:46691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:40555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:40555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:59.346164 30957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:59.347692 30957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:59.361155 30965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:59.361102 30962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:59.361416 30963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:59.362592 30957 server_base.cc:1034] running on GCE node
I20260806 01:13:59.364028 30957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:59.365764 30957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:59.367048 30957 hybrid_clock.cc:648] HybridClock initialized: now 1785978839366999 us; error 167 us; skew 500 ppm
I20260806 01:13:59.367642 30957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:59 dist-test-slave-7nm7 krb5kdc[30934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978839, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:59.371340 30957 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:13:59.373230 30957 webserver.cc:460] Webserver started at http://127.25.236.190:35729/ using document root <none> and password file <none>
I20260806 01:13:59.374049 30957 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:59.374220 30957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:59.374615 30957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:59.377660 30957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "608b311dce274bfc94521aaa4148a9f9"
format_stamp: "Formatted at 2026-08-06 01:13:59 on dist-test-slave-7nm7"
server_key: "82599a3c03d3dcd3474659d54ec1ab14"
server_key_iv: "dd845c48b8d190c1f06606aa95176714"
server_key_version: "encryptionkey@0"
I20260806 01:13:59.378566 30957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "608b311dce274bfc94521aaa4148a9f9"
format_stamp: "Formatted at 2026-08-06 01:13:59 on dist-test-slave-7nm7"
server_key: "82599a3c03d3dcd3474659d54ec1ab14"
server_key_iv: "dd845c48b8d190c1f06606aa95176714"
server_key_version: "encryptionkey@0"
I20260806 01:13:59.384577 30957 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 01:13:59.388607 30976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:59.390570 30957 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 01:13:59.390799 30957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "608b311dce274bfc94521aaa4148a9f9"
format_stamp: "Formatted at 2026-08-06 01:13:59 on dist-test-slave-7nm7"
server_key: "82599a3c03d3dcd3474659d54ec1ab14"
server_key_iv: "dd845c48b8d190c1f06606aa95176714"
server_key_version: "encryptionkey@0"
I20260806 01:13:59.391368 30957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:59.416428 30957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:59.420913 30957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:59.421175 30957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:59.442183 30957 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:40555
I20260806 01:13:59.442207 31056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:40555 every 8 connection(s)
I20260806 01:13:59.443758 30957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:13:59.447821 31057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:59.449828 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 30957
I20260806 01:13:59.450093 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:13:59.451025 26546 external_mini_cluster.cc:1428] Setting key a873b01629f9f6f96d6c73ff64eb813e
I20260806 01:13:59.462062 31057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9: Bootstrap starting.
I20260806 01:13:59.466657 31057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9: Neither blocks nor log segments found. Creating new log.
Aug 06 01:13:59 dist-test-slave-7nm7 krb5kdc[30934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:13:59.467911 31057 log.cc:824] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:59.471778 31057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9: No bootstrap required, opened a new log
I20260806 01:13:59.478798 31062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:59.453254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36778 (local address 127.25.236.190:40555)
0806 01:13:59.454210 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:59.454284 (+ 74us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:59.454341 (+ 57us) server_negotiation.cc:407] Connection header received
0806 01:13:59.456027 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:59.456049 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:59.456433 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:59.456752 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:59.458994 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:59.460482 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:59.462547 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:59.463223 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:59.463359 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:59.468684 (+ 5325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:59.468697 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:59.468722 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:59.468774 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:59.472963 (+ 4189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:59.473796 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:59.473806 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:59.473821 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:59.474059 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:59.474604 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:59.474610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:59.474613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:59.475471 (+ 858us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:59.475853 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:59.476314 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 01:13:59.476849 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":417,"threads_started":1}
I20260806 01:13:59.484454 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:13:59.484568 31057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608b311dce274bfc94521aaa4148a9f9" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40555 } }
I20260806 01:13:59.485051 31057 raft_consensus.cc:374] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:59.485160 31057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 608b311dce274bfc94521aaa4148a9f9, State: Initialized, Role: FOLLOWER
I20260806 01:13:59.485800 31057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608b311dce274bfc94521aaa4148a9f9" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40555 } }
I20260806 01:13:59.486061 31057 raft_consensus.cc:388] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:13:59.486213 31057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:13:59.486472 31057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:59.488894 31057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608b311dce274bfc94521aaa4148a9f9" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40555 } }
I20260806 01:13:59.489519 31057 leader_election.cc:302] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 608b311dce274bfc94521aaa4148a9f9; no voters:
I20260806 01:13:59.490285 31057 leader_election.cc:288] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:13:59.490435 31064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:59.493803 31057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:59.494529 31064 raft_consensus.cc:686] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [term 1 LEADER]: Becoming Leader. State: Replica: 608b311dce274bfc94521aaa4148a9f9, State: Running, Role: LEADER
I20260806 01:13:59.495364 31064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608b311dce274bfc94521aaa4148a9f9" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40555 } }
I20260806 01:13:59.499519 31065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "608b311dce274bfc94521aaa4148a9f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608b311dce274bfc94521aaa4148a9f9" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40555 } } }
I20260806 01:13:59.500340 31065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:59.499308 31067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 608b311dce274bfc94521aaa4148a9f9. Latest consensus state: current_term: 1 leader_uuid: "608b311dce274bfc94521aaa4148a9f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "608b311dce274bfc94521aaa4148a9f9" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40555 } } }
I20260806 01:13:59.501041 31067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:59.515409 31073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:13:59.521149 31073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:13:59.545369 31073 catalog_manager.cc:1347] Generated new cluster ID: 5cd4d548e77144b1ba41cc160d53ff9a
I20260806 01:13:59.545542 31073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:13:59.569255 31073 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:13:59.571977 31073 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:13:59.579487 31099 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 01:13:59.580199 31073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 608b311dce274bfc94521aaa4148a9f9: Generated new TSK 0
I20260806 01:13:59.581269 31073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:13:59.583947 31062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:59.567671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36782 (local address 127.25.236.190:40555)
0806 01:13:59.567914 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:59.567924 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:59.575035 (+ 7111us) server_negotiation.cc:407] Connection header received
0806 01:13:59.575483 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:59.575490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:59.575576 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:59.575641 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:59.577063 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:59.578332 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:59.580403 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:59.580917 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:59.580991 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:59.582392 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:59.582397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:59.582400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:13:59.583135 (+ 735us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:59.583196 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:13:59.583225 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:59.583364 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:59.583501 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 01:13:59.583602 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":50}
I20260806 01:13:59.591917 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 30957
I20260806 01:13:59.593228 31048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:59.696856 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 30957
2026-08-06T01:13:59Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2369 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:13:59 dist-test-slave-7nm7 krb5kdc[31103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:13:59 dist-test-slave-7nm7 krb5kdc[31103](info): set up 2 sockets
Aug 06 01:13:59 dist-test-slave-7nm7 krb5kdc[31103](info): commencing operation
krb5kdc: starting...
W20260806 01:14:01.092434 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.343s user 0.008s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:14:01 dist-test-slave-7nm7 krb5kdc[31103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:14:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:14:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:14:01.364548 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:34761
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:44155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:34761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:01.750046 31124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:01.750427 31124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:01.750689 31124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:01.760908 31124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:01.761050 31124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:01.761165 31124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:01.761245 31124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:01.761336 31124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:14:01.773641 31124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:44155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:34761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:34761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:01.776482 31124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:01.778651 31124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:01.794553 31132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:01.794986 31135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:01.794960 31133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:01.796173 31124 server_base.cc:1034] running on GCE node
I20260806 01:14:01.797075 31124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:01.799809 31124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:01.801062 31124 hybrid_clock.cc:648] HybridClock initialized: now 1785978841801004 us; error 49 us; skew 500 ppm
I20260806 01:14:01.801520 31124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:01 dist-test-slave-7nm7 krb5kdc[31103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978841, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:01.805428 31124 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:14:01.807189 31124 webserver.cc:460] Webserver started at http://127.25.236.190:43487/ using document root <none> and password file <none>
I20260806 01:14:01.807969 31124 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:01.808112 31124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:01.808490 31124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:01.811717 31124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "4835639a677c4ff6ab5be05600226a77"
format_stamp: "Formatted at 2026-08-06 01:14:01 on dist-test-slave-7nm7"
server_key: "935780c9703ea20eb4330281eed6fb24"
server_key_iv: "7218b8d930e88addb08f02153f15ca6e"
server_key_version: "encryptionkey@0"
I20260806 01:14:01.812711 31124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "4835639a677c4ff6ab5be05600226a77"
format_stamp: "Formatted at 2026-08-06 01:14:01 on dist-test-slave-7nm7"
server_key: "935780c9703ea20eb4330281eed6fb24"
server_key_iv: "7218b8d930e88addb08f02153f15ca6e"
server_key_version: "encryptionkey@0"
I20260806 01:14:01.818423 31124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 01:14:01.822755 31144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:01.824599 31124 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260806 01:14:01.824867 31124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "4835639a677c4ff6ab5be05600226a77"
format_stamp: "Formatted at 2026-08-06 01:14:01 on dist-test-slave-7nm7"
server_key: "935780c9703ea20eb4330281eed6fb24"
server_key_iv: "7218b8d930e88addb08f02153f15ca6e"
server_key_version: "encryptionkey@0"
I20260806 01:14:01.825125 31124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:01.879858 31124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:01.885792 31124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:01.886173 31124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:01.913229 31124 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:34761
I20260806 01:14:01.913280 31219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:34761 every 8 connection(s)
I20260806 01:14:01.914824 31124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:14:01.920053 31221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:01.920953 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 31124
I20260806 01:14:01.921289 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:14:01.922060 26546 external_mini_cluster.cc:1428] Setting key b97daae35a1488249e1928abc4fcd10e
I20260806 01:14:01.937480 31221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77: Bootstrap starting.
Aug 06 01:14:01 dist-test-slave-7nm7 krb5kdc[31103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:01.941094 31221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:01.942318 31221 log.cc:824] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:01.945772 31221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77: No bootstrap required, opened a new log
I20260806 01:14:01.950037 31224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:01.925010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49282 (local address 127.25.236.190:34761)
0806 01:14:01.925893 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:01.925915 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:01.925972 (+ 57us) server_negotiation.cc:407] Connection header received
0806 01:14:01.927906 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:01.927938 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:01.928497 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:01.928971 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:01.931524 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:01.933905 (+ 2381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:01.935452 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:01.936686 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:01.936891 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:01.941472 (+ 4581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:01.941481 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:01.941499 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:01.941550 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:01.945087 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:01.946150 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:01.946157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:01.946163 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:01.946409 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:01.946846 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:01.946852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:01.946855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:01.947556 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:01.947962 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:01.948540 (+ 578us) server_negotiation.cc:299] Negotiation successful
0806 01:14:01.948992 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":354,"threads_started":1}
I20260806 01:14:01.955763 31221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4835639a677c4ff6ab5be05600226a77" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34761 } }
I20260806 01:14:01.956218 31221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:01.956341 31221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4835639a677c4ff6ab5be05600226a77, State: Initialized, Role: FOLLOWER
I20260806 01:14:01.956903 31221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4835639a677c4ff6ab5be05600226a77" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34761 } }
I20260806 01:14:01.957149 31221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:14:01.957311 31221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:14:01.957504 31221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:01.959838 31221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4835639a677c4ff6ab5be05600226a77" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34761 } }
I20260806 01:14:01.960366 31221 leader_election.cc:302] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4835639a677c4ff6ab5be05600226a77; no voters:
I20260806 01:14:01.961055 31221 leader_election.cc:288] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:14:01.961297 31226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:01.962181 31226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [term 1 LEADER]: Becoming Leader. State: Replica: 4835639a677c4ff6ab5be05600226a77, State: Running, Role: LEADER
I20260806 01:14:01.962929 31226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4835639a677c4ff6ab5be05600226a77" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34761 } }
I20260806 01:14:01.963567 31221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:01.966056 31231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4835639a677c4ff6ab5be05600226a77. Latest consensus state: current_term: 1 leader_uuid: "4835639a677c4ff6ab5be05600226a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4835639a677c4ff6ab5be05600226a77" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34761 } } }
I20260806 01:14:01.966115 31227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4835639a677c4ff6ab5be05600226a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4835639a677c4ff6ab5be05600226a77" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34761 } } }
I20260806 01:14:01.966550 31231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:01.966550 31227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:01.972370 31242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:01.978492 31242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:01.988965 31242 catalog_manager.cc:1347] Generated new cluster ID: a72bcd0230da41799b2f7d2ee246cd87
I20260806 01:14:01.989406 31242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:02.001036 31242 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:14:02.002754 31242 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:02.015579 31242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4835639a677c4ff6ab5be05600226a77: Generated new TSK 0
I20260806 01:14:02.016486 31242 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:14:02.082022 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34761
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:44155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:02.469218 31255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:02.469559 31255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:02.469743 31255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:02.478188 31255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:02.478442 31255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:02.478641 31255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:14:02.489666 31255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:44155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:02.491252 31255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:02.493147 31255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:02.508967 31263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:02.508998 31264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:02.509212 31268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:02.511361 31255 server_base.cc:1034] running on GCE node
I20260806 01:14:02.511996 31255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:02.513190 31255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:02.514451 31255 hybrid_clock.cc:648] HybridClock initialized: now 1785978842514385 us; error 59 us; skew 500 ppm
I20260806 01:14:02.514803 31255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:02 dist-test-slave-7nm7 krb5kdc[31103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978842, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:02.519150 31255 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:14:02.521848 31255 webserver.cc:460] Webserver started at http://127.25.236.129:36547/ using document root <none> and password file <none>
I20260806 01:14:02.522707 31255 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:02.522869 31255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:02.523200 31255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:02.526327 31255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "9b90620a2484437583338cdb39623171"
format_stamp: "Formatted at 2026-08-06 01:14:02 on dist-test-slave-7nm7"
server_key: "bc5522bf70144bad3b31c7bcb1e27259"
server_key_iv: "982f819ddb87d1bdc0ae9f3e0c1c1e65"
server_key_version: "encryptionkey@0"
I20260806 01:14:02.527154 31255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "9b90620a2484437583338cdb39623171"
format_stamp: "Formatted at 2026-08-06 01:14:02 on dist-test-slave-7nm7"
server_key: "bc5522bf70144bad3b31c7bcb1e27259"
server_key_iv: "982f819ddb87d1bdc0ae9f3e0c1c1e65"
server_key_version: "encryptionkey@0"
I20260806 01:14:02.533324 31255 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260806 01:14:02.537196 31279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:02.538720 31255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:14:02.538924 31255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "9b90620a2484437583338cdb39623171"
format_stamp: "Formatted at 2026-08-06 01:14:02 on dist-test-slave-7nm7"
server_key: "bc5522bf70144bad3b31c7bcb1e27259"
server_key_iv: "982f819ddb87d1bdc0ae9f3e0c1c1e65"
server_key_version: "encryptionkey@0"
I20260806 01:14:02.539170 31255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:02.552759 31255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:02.557925 31255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:02.558346 31255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:02.559633 31255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:02.561301 31255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:02.561411 31255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:02.561575 31255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:02.561650 31255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:02.601521 31255 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:40031
I20260806 01:14:02.601565 31443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:40031 every 8 connection(s)
I20260806 01:14:02.603042 31255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:14:02.610931 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 31255
I20260806 01:14:02.611166 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:14:02.612035 26546 external_mini_cluster.cc:1428] Setting key 967f08955a3e6187111bed969bc85873
Aug 06 01:14:02 dist-test-slave-7nm7 krb5kdc[31103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978842, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:02.632329 31446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:02.605887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:44661 (local address 127.25.236.190:34761)
0806 01:14:02.606604 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:02.606611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:02.607911 (+ 1300us) server_negotiation.cc:407] Connection header received
0806 01:14:02.611484 (+ 3573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:02.611493 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:02.611594 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:02.611735 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:02.614962 (+ 3227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:02.616475 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:02.619937 (+ 3462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:02.620703 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:02.620887 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:02.625832 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:02.625838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:02.625841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:02.625874 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:02.628732 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:02.629493 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:02.629499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:02.629502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:02.629627 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:02.630201 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:02.630206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:02.630210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:02.630523 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:02.630695 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:02.631568 (+ 873us) server_negotiation.cc:299] Negotiation successful
0806 01:14:02.631799 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":417,"threads_started":1}
I20260806 01:14:02.633361 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:02.606513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34761 (local address 127.25.236.129:44661)
0806 01:14:02.607688 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:02.607751 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:02.611108 (+ 3357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:02.612089 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:02.612105 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:02.613008 (+ 903us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:02.614743 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:02.614775 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:02.616855 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:02.616954 (+ 99us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:02.619681 (+ 2727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:02.619706 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:02.621273 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:02.621283 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:02.621624 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:02.622562 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:02.622604 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:02.625572 (+ 2968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:02.628917 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:02.628926 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:02.628942 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:02.629360 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:02.629805 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:02.629809 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:02.629811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:02.630078 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:02.630660 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:02.630667 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:02.631142 (+ 475us) client_negotiation.cc:769] Sending connection context
0806 01:14:02.631442 (+ 300us) client_negotiation.cc:240] Negotiation successful
0806 01:14:02.631989 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"thread_start_us":487,"threads_started":1}
I20260806 01:14:02.635413 31444 heartbeater.cc:347] Connected to a master server at 127.25.236.190:34761
I20260806 01:14:02.635748 31444 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:02.636540 31444 heartbeater.cc:511] Master 127.25.236.190:34761 requested a full tablet report, sending...
I20260806 01:14:02.638973 31167 ts_manager.cc:194] Registered new tserver with Master: 9b90620a2484437583338cdb39623171 (127.25.236.129:40031)
I20260806 01:14:02.640825 31167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:44661
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:14:02.681824 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34761
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:44155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:03.069160 31450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:03.069449 31450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:03.069563 31450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:03.079821 31450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:03.079954 31450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:03.080072 31450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:14:03.090775 31450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:44155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:03.092159 31450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:03.093539 31450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:03.107755 31459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:03.107883 31461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:03.107883 31464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:03.108763 31450 server_base.cc:1034] running on GCE node
I20260806 01:14:03.109421 31450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:03.110579 31450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:03.111827 31450 hybrid_clock.cc:648] HybridClock initialized: now 1785978843111773 us; error 46 us; skew 500 ppm
I20260806 01:14:03.112214 31450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:03 dist-test-slave-7nm7 krb5kdc[31103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978843, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:03.115947 31450 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:14:03.117969 31450 webserver.cc:460] Webserver started at http://127.25.236.130:40193/ using document root <none> and password file <none>
I20260806 01:14:03.118788 31450 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:03.118942 31450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:03.119272 31450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:03.122364 31450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "dfb21816ad3b45aa89210b8e4a08af15"
format_stamp: "Formatted at 2026-08-06 01:14:03 on dist-test-slave-7nm7"
server_key: "5c952ea751f340ee310dd5c4216676a8"
server_key_iv: "1b58e1895e99b4e3717d98c8d792d5a1"
server_key_version: "encryptionkey@0"
I20260806 01:14:03.123163 31450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "dfb21816ad3b45aa89210b8e4a08af15"
format_stamp: "Formatted at 2026-08-06 01:14:03 on dist-test-slave-7nm7"
server_key: "5c952ea751f340ee310dd5c4216676a8"
server_key_iv: "1b58e1895e99b4e3717d98c8d792d5a1"
server_key_version: "encryptionkey@0"
I20260806 01:14:03.129130 31450 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 01:14:03.133648 31472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:03.135581 31450 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:14:03.135809 31450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "dfb21816ad3b45aa89210b8e4a08af15"
format_stamp: "Formatted at 2026-08-06 01:14:03 on dist-test-slave-7nm7"
server_key: "5c952ea751f340ee310dd5c4216676a8"
server_key_iv: "1b58e1895e99b4e3717d98c8d792d5a1"
server_key_version: "encryptionkey@0"
I20260806 01:14:03.136039 31450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:03.161375 31450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:03.166852 31450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:03.167173 31450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:03.168704 31450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:03.170326 31450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:03.170449 31450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:03.170583 31450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:03.170667 31450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:03.211727 31450 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:37661
I20260806 01:14:03.211778 31661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:37661 every 8 connection(s)
I20260806 01:14:03.213114 31450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:14:03.222579 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 31450
I20260806 01:14:03.222769 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:14:03.223227 26546 external_mini_cluster.cc:1428] Setting key 76bf048d7bd96ac41b27ffee0b4c5c82
Aug 06 01:14:03 dist-test-slave-7nm7 krb5kdc[31103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978843, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:03.239398 31665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:03.215965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:44403 (local address 127.25.236.190:34761)
0806 01:14:03.216566 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:03.216571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:03.217743 (+ 1172us) server_negotiation.cc:407] Connection header received
0806 01:14:03.220520 (+ 2777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:03.220525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:03.220618 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:03.220737 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:03.222365 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.223307 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.225317 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.225787 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.225920 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:03.231155 (+ 5235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:03.231175 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:03.231180 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:03.231224 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:03.234968 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:03.235932 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:03.235939 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:03.235944 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:03.236129 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:03.236634 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:03.236641 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:03.236644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:03.237071 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:03.237245 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:03.238615 (+ 1370us) server_negotiation.cc:299] Negotiation successful
0806 01:14:03.238903 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":378,"threads_started":1}
I20260806 01:14:03.240314 31667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:03.216534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34761 (local address 127.25.236.130:44403)
0806 01:14:03.217484 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:03.217576 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:03.220162 (+ 2586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:03.220952 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:03.220960 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:03.221474 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:03.222045 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:03.222062 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.223521 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.223527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:03.225159 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:03.225177 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.226064 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.226071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:03.226341 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:03.227102 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:03.227133 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:03.230787 (+ 3654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:03.235159 (+ 4372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:03.235173 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:03.235197 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:03.235739 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:03.236272 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:03.236277 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:03.236279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:03.236486 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:03.237247 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:03.237254 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:03.238066 (+ 812us) client_negotiation.cc:769] Sending connection context
0806 01:14:03.238461 (+ 395us) client_negotiation.cc:240] Negotiation successful
0806 01:14:03.238911 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":269,"threads_started":1}
I20260806 01:14:03.241988 31662 heartbeater.cc:347] Connected to a master server at 127.25.236.190:34761
I20260806 01:14:03.242460 31662 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:03.243330 31662 heartbeater.cc:511] Master 127.25.236.190:34761 requested a full tablet report, sending...
I20260806 01:14:03.245007 31167 ts_manager.cc:194] Registered new tserver with Master: dfb21816ad3b45aa89210b8e4a08af15 (127.25.236.130:37661)
I20260806 01:14:03.246053 31167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:44403
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:14:03.286000 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34761
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:44155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:14:03.644584 31444 heartbeater.cc:503] Master 127.25.236.190:34761 was elected leader, sending a full tablet report...
W20260806 01:14:03.665683 31672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:03.666043 31672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:03.666273 31672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:03.674355 31672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:03.674535 31672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:03.674737 31672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:14:03.686136 31672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:44155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:03.687920 31672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:03.689316 31672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:03.703608 31683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:03.703603 31686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:03.703595 31684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:03.703897 31672 server_base.cc:1034] running on GCE node
I20260806 01:14:03.705165 31672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:03.706108 31672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:03.707345 31672 hybrid_clock.cc:648] HybridClock initialized: now 1785978843707287 us; error 45 us; skew 500 ppm
I20260806 01:14:03.707695 31672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:03 dist-test-slave-7nm7 krb5kdc[31103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978843, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:03.710985 31672 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:14:03.712626 31672 webserver.cc:460] Webserver started at http://127.25.236.131:39935/ using document root <none> and password file <none>
I20260806 01:14:03.713405 31672 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:03.713574 31672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:03.713915 31672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:03.717335 31672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "94d900f86fae405b8c0d947159ba9f8e"
format_stamp: "Formatted at 2026-08-06 01:14:03 on dist-test-slave-7nm7"
server_key: "d3c04c0cc9933c9ccd19931d2ffefe01"
server_key_iv: "df765dfa8d7603b54a9afc4945ce76c5"
server_key_version: "encryptionkey@0"
I20260806 01:14:03.718173 31672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "94d900f86fae405b8c0d947159ba9f8e"
format_stamp: "Formatted at 2026-08-06 01:14:03 on dist-test-slave-7nm7"
server_key: "d3c04c0cc9933c9ccd19931d2ffefe01"
server_key_iv: "df765dfa8d7603b54a9afc4945ce76c5"
server_key_version: "encryptionkey@0"
I20260806 01:14:03.723577 31672 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:14:03.727216 31695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:03.728616 31672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 01:14:03.728801 31672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "94d900f86fae405b8c0d947159ba9f8e"
format_stamp: "Formatted at 2026-08-06 01:14:03 on dist-test-slave-7nm7"
server_key: "d3c04c0cc9933c9ccd19931d2ffefe01"
server_key_iv: "df765dfa8d7603b54a9afc4945ce76c5"
server_key_version: "encryptionkey@0"
I20260806 01:14:03.729017 31672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:03.753791 31672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:03.760363 31672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:03.760731 31672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:03.762409 31672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:03.764084 31672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:03.764215 31672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:03.764375 31672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:03.764449 31672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:03.807534 31672 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:33235
I20260806 01:14:03.807566 31867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:33235 every 8 connection(s)
I20260806 01:14:03.808979 31672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:14:03.815613 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 31672
I20260806 01:14:03.815819 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:14:03.816392 26546 external_mini_cluster.cc:1428] Setting key f9ea6626e3b916b6e733b93705d4d42b
Aug 06 01:14:03 dist-test-slave-7nm7 krb5kdc[31103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978843, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:03.834419 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:03.811900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:58699 (local address 127.25.236.190:34761)
0806 01:14:03.812492 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:03.812496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:03.813686 (+ 1190us) server_negotiation.cc:407] Connection header received
0806 01:14:03.816801 (+ 3115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:03.816806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:03.816899 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:03.817038 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:03.819210 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.820512 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.822858 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.823263 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.823412 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:03.827204 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:03.827209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:03.827213 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:03.827246 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:03.830674 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:03.831530 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:03.831537 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:03.831547 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:03.831682 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:03.832190 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:03.832203 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:03.832206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:03.832545 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:03.832736 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:03.833676 (+ 940us) server_negotiation.cc:299] Negotiation successful
0806 01:14:03.833945 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":366,"threads_started":1}
I20260806 01:14:03.835433 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:03.812354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34761 (local address 127.25.236.131:58699)
0806 01:14:03.813496 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:03.813545 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:03.816445 (+ 2900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:03.817698 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:03.817708 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:03.818370 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:03.819002 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:03.819019 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.820702 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.820707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:03.822656 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:03.822671 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.823517 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.823525 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:03.823785 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:03.824526 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:03.824551 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:03.826939 (+ 2388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:03.830860 (+ 3921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:03.830871 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:03.830893 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:03.831376 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:03.831814 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:03.831820 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:03.831824 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:03.832048 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:03.832760 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:03.832770 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:03.833192 (+ 422us) client_negotiation.cc:769] Sending connection context
0806 01:14:03.833542 (+ 350us) client_negotiation.cc:240] Negotiation successful
0806 01:14:03.834145 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"thread_start_us":465,"threads_started":1}
I20260806 01:14:03.836916 31871 heartbeater.cc:347] Connected to a master server at 127.25.236.190:34761
I20260806 01:14:03.837270 31871 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:03.838011 31871 heartbeater.cc:511] Master 127.25.236.190:34761 requested a full tablet report, sending...
I20260806 01:14:03.839776 31167 ts_manager.cc:194] Registered new tserver with Master: 94d900f86fae405b8c0d947159ba9f8e (127.25.236.131:33235)
I20260806 01:14:03.840660 31167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:58699
I20260806 01:14:03.845314 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:14:03 dist-test-slave-7nm7 krb5kdc[31103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978843, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 01:14:03 dist-test-slave-7nm7 krb5kdc[31103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978843, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:03.908876 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:03.892527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49292 (local address 127.25.236.190:34761)
0806 01:14:03.892815 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:03.892822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:03.893217 (+ 395us) server_negotiation.cc:407] Connection header received
0806 01:14:03.893488 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:03.893493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:03.893588 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:03.893701 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:03.896039 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.896872 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.898484 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.898900 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.899014 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:03.903042 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:03.903047 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:03.903051 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:03.903091 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:03.906027 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:03.906774 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:03.906778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:03.906780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:03.906913 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:03.907349 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:03.907353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:03.907355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:03.907710 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:03.907908 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:03.908325 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 01:14:03.908502 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260806 01:14:03.909543 31167 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49292
I20260806 01:14:03.911899 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 31255
Aug 06 01:14:03 dist-test-slave-7nm7 krb5kdc[31103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978843, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:14:03.931718 31888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:03.913237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44242 (local address 127.25.236.129:40031)
0806 01:14:03.914810 (+ 1573us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:03.914817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:03.914843 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:14:03.914902 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:03.914908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:03.915123 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:03.915325 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:03.916611 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.917619 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.919465 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.920389 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.920603 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:03.925514 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:03.925527 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:03.925542 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:03.925576 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:03.928753 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:03.929504 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:03.929511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:03.929513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:03.929645 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:03.930054 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:03.930059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:03.930062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:03.930491 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:03.930819 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:03.931070 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 01:14:03.931336 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1420,"thread_start_us":1097,"threads_started":1}
W20260806 01:14:03.933017 31437 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44242
W20260806 01:14:03.934336 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 31255: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:14:03.934583 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 31255
I20260806 01:14:03.967707 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 31450
Aug 06 01:14:03 dist-test-slave-7nm7 krb5kdc[31103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978843, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:14:03.986182 31889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:03.968732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33664 (local address 127.25.236.130:37661)
0806 01:14:03.969277 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:03.969285 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:03.969311 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:14:03.969371 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:03.969377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:03.969590 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:03.969727 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:03.971062 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.972301 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.973690 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:03.974379 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:03.974490 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:03.979471 (+ 4981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:03.979487 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:03.979505 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:03.979556 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:03.982977 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:03.983725 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:03.983733 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:03.983735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:03.983866 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:03.984265 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:03.984272 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:03.984279 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:03.984627 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:03.984967 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:03.985414 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 01:14:03.985710 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":270,"threads_started":1}
W20260806 01:14:03.988335 31653 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33664
W20260806 01:14:03.989434 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 31450: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:14:03.989616 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 31450
I20260806 01:14:04.031772 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 31672
Aug 06 01:14:04 dist-test-slave-7nm7 krb5kdc[31103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978843, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:14:04.049101 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:04.032988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58700 (local address 127.25.236.131:33235)
0806 01:14:04.033839 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:04.033846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:04.033878 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:14:04.033939 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:04.033945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:04.034159 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:04.034500 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:04.035826 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:04.036917 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:04.038099 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:04.038755 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:04.038858 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:04.043364 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:04.043377 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:04.043392 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:04.043434 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:04.046463 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:04.047087 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:04.047094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:04.047096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:04.047219 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:04.047548 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:04.047551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:04.047553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:04.047934 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:04.048176 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:04.048486 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 01:14:04.048744 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":392,"threads_started":1}
W20260806 01:14:04.050428 31862 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58700
W20260806 01:14:04.051522 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 31672: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:14:04.051808 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 31672
I20260806 01:14:04.076573 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 31124
I20260806 01:14:04.077984 31210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:04.187171 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 31124
2026-08-06T01:14:04Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4493 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:14:04 dist-test-slave-7nm7 krb5kdc[31894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:14:04 dist-test-slave-7nm7 krb5kdc[31894](info): set up 2 sockets
Aug 06 01:14:04 dist-test-slave-7nm7 krb5kdc[31894](info): commencing operation
krb5kdc: starting...
W20260806 01:14:05.557366 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:14:05 dist-test-slave-7nm7 krb5kdc[31894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:14:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:14:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:14:05.831241 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:46091
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:33359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:46091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:06.173151 31914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:06.173465 31914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:06.173640 31914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:06.181675 31914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:06.181807 31914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:06.181910 31914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:06.182039 31914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:06.182109 31914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:14:06.192759 31914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:33359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:46091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:46091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:06.194454 31914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:06.195854 31914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:06.207981 31920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:06.208041 31929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:06.207965 31924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:06.209656 31914 server_base.cc:1034] running on GCE node
I20260806 01:14:06.210728 31914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:06.212646 31914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:06.213912 31914 hybrid_clock.cc:648] HybridClock initialized: now 1785978846213861 us; error 38 us; skew 500 ppm
I20260806 01:14:06.214491 31914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:06 dist-test-slave-7nm7 krb5kdc[31894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978846, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:06.218348 31914 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:14:06.220337 31914 webserver.cc:460] Webserver started at http://127.25.236.190:42573/ using document root <none> and password file <none>
I20260806 01:14:06.221130 31914 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:06.221284 31914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:06.221639 31914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:06.225028 31914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "9eedb7802491495e984a7bd4f65c22e0"
format_stamp: "Formatted at 2026-08-06 01:14:06 on dist-test-slave-7nm7"
server_key: "ff35f7d6971ecd4819d261e12d63de31"
server_key_iv: "f905391634d1d6c23a9b3cbab94d59cd"
server_key_version: "encryptionkey@0"
I20260806 01:14:06.225893 31914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "9eedb7802491495e984a7bd4f65c22e0"
format_stamp: "Formatted at 2026-08-06 01:14:06 on dist-test-slave-7nm7"
server_key: "ff35f7d6971ecd4819d261e12d63de31"
server_key_iv: "f905391634d1d6c23a9b3cbab94d59cd"
server_key_version: "encryptionkey@0"
I20260806 01:14:06.231506 31914 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:14:06.235780 31937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:06.237386 31914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:14:06.237615 31914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "9eedb7802491495e984a7bd4f65c22e0"
format_stamp: "Formatted at 2026-08-06 01:14:06 on dist-test-slave-7nm7"
server_key: "ff35f7d6971ecd4819d261e12d63de31"
server_key_iv: "f905391634d1d6c23a9b3cbab94d59cd"
server_key_version: "encryptionkey@0"
I20260806 01:14:06.237841 31914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:06.255569 31914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:06.260349 31914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:06.260668 31914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:06.280470 31914 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:46091
I20260806 01:14:06.280494 32017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:46091 every 8 connection(s)
I20260806 01:14:06.281853 31914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:14:06.285669 32018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:06.286571 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 31914
I20260806 01:14:06.286814 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:14:06.287470 26546 external_mini_cluster.cc:1428] Setting key d51fddfcbd34e76233f84bcb0749f41b
I20260806 01:14:06.300757 32018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0: Bootstrap starting.
Aug 06 01:14:06 dist-test-slave-7nm7 krb5kdc[31894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:06.305109 32018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:06.307042 32018 log.cc:824] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:06.310691 32018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0: No bootstrap required, opened a new log
I20260806 01:14:06.312443 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:06.289500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37650 (local address 127.25.236.190:46091)
0806 01:14:06.290365 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:06.290377 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:06.290422 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:14:06.292119 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:06.292141 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:06.292539 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:06.292887 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:06.294806 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:06.296102 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:06.298059 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:06.298841 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:06.298977 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:06.304360 (+ 5383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:06.304369 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:06.304384 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:06.304429 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:06.307877 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:06.308730 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:06.308737 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:06.308744 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:06.308910 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:06.309373 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:06.309377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:06.309379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:06.309919 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:06.310157 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:06.310687 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 01:14:06.311300 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":380,"threads_started":1}
I20260806 01:14:06.318464 32018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eedb7802491495e984a7bd4f65c22e0" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46091 } }
I20260806 01:14:06.318809 32018 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:06.318902 32018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eedb7802491495e984a7bd4f65c22e0, State: Initialized, Role: FOLLOWER
I20260806 01:14:06.319458 32018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eedb7802491495e984a7bd4f65c22e0" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46091 } }
I20260806 01:14:06.319697 32018 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:14:06.319832 32018 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:14:06.320003 32018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:06.322400 32018 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eedb7802491495e984a7bd4f65c22e0" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46091 } }
I20260806 01:14:06.322968 32018 leader_election.cc:302] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9eedb7802491495e984a7bd4f65c22e0; no voters:
I20260806 01:14:06.323560 32018 leader_election.cc:288] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:14:06.323740 32027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:06.324488 32027 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [term 1 LEADER]: Becoming Leader. State: Replica: 9eedb7802491495e984a7bd4f65c22e0, State: Running, Role: LEADER
I20260806 01:14:06.325029 32027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eedb7802491495e984a7bd4f65c22e0" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46091 } }
I20260806 01:14:06.325749 32018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:06.328161 32029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9eedb7802491495e984a7bd4f65c22e0. Latest consensus state: current_term: 1 leader_uuid: "9eedb7802491495e984a7bd4f65c22e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eedb7802491495e984a7bd4f65c22e0" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46091 } } }
I20260806 01:14:06.328502 32029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:06.327998 32028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9eedb7802491495e984a7bd4f65c22e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eedb7802491495e984a7bd4f65c22e0" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46091 } } }
I20260806 01:14:06.329684 32028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:06.334647 32037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:06.340304 32037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:06.349332 32037 catalog_manager.cc:1347] Generated new cluster ID: 20bb4e4085304a2f8e248a5572009883
I20260806 01:14:06.349458 32037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:06.378970 32037 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:14:06.381026 32037 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:06.388468 32037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9eedb7802491495e984a7bd4f65c22e0: Generated new TSK 0
I20260806 01:14:06.389456 32037 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:14:06.439375 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:46091
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:33359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:06.815120 32063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:06.815445 32063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:06.815629 32063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:06.823477 32063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:06.823606 32063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:06.823798 32063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:14:06.835237 32063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:33359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:46091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:06.836632 32063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:06.838044 32063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:06.853235 32071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:06.853364 32070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:06.853361 32074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:06.854605 32063 server_base.cc:1034] running on GCE node
I20260806 01:14:06.855288 32063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:06.856508 32063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:06.857934 32063 hybrid_clock.cc:648] HybridClock initialized: now 1785978846857840 us; error 51 us; skew 500 ppm
I20260806 01:14:06.858594 32063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:06 dist-test-slave-7nm7 krb5kdc[31894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978846, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:06.866849 32063 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:14:06.868572 32063 webserver.cc:460] Webserver started at http://127.25.236.129:45633/ using document root <none> and password file <none>
I20260806 01:14:06.869321 32063 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:06.869472 32063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:06.869808 32063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:06.872952 32063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "a267c4d1c23a440abf9fffcb3c72a368"
format_stamp: "Formatted at 2026-08-06 01:14:06 on dist-test-slave-7nm7"
server_key: "1bdce066cf8312228c49a0a1ee11847f"
server_key_iv: "e3ffac18dce79a0cf9a5b57cb75f3682"
server_key_version: "encryptionkey@0"
I20260806 01:14:06.874516 32063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "a267c4d1c23a440abf9fffcb3c72a368"
format_stamp: "Formatted at 2026-08-06 01:14:06 on dist-test-slave-7nm7"
server_key: "1bdce066cf8312228c49a0a1ee11847f"
server_key_iv: "e3ffac18dce79a0cf9a5b57cb75f3682"
server_key_version: "encryptionkey@0"
I20260806 01:14:06.881068 32063 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 01:14:06.885036 32084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:06.886687 32063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:14:06.886885 32063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "a267c4d1c23a440abf9fffcb3c72a368"
format_stamp: "Formatted at 2026-08-06 01:14:06 on dist-test-slave-7nm7"
server_key: "1bdce066cf8312228c49a0a1ee11847f"
server_key_iv: "e3ffac18dce79a0cf9a5b57cb75f3682"
server_key_version: "encryptionkey@0"
I20260806 01:14:06.887120 32063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:06.909363 32063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:06.915900 32063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:06.916467 32063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:06.917980 32063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:06.919821 32063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:06.919945 32063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:06.920123 32063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:06.920207 32063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:06.962890 32063 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:35489
I20260806 01:14:06.962934 32250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:35489 every 8 connection(s)
I20260806 01:14:06.964498 32063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:14:06.966884 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 32063
I20260806 01:14:06.967236 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:14:06.968165 26546 external_mini_cluster.cc:1428] Setting key 31f6ca4ce5a93808a6638a8bc43bae55
Aug 06 01:14:06 dist-test-slave-7nm7 krb5kdc[31894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978846, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:06.992764 32253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:06.967763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:40521 (local address 127.25.236.190:46091)
0806 01:14:06.968648 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:06.968658 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:06.969943 (+ 1285us) server_negotiation.cc:407] Connection header received
0806 01:14:06.972620 (+ 2677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:06.972626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:06.972722 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:06.972864 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:06.974684 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:06.975472 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:06.977171 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:06.977603 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:06.977760 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:06.982805 (+ 5045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:06.982814 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:06.982821 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:06.982874 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:06.987634 (+ 4760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:06.988984 (+ 1350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:06.988994 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:06.989000 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:06.989460 (+ 460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:06.989976 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:06.989984 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:06.989990 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:06.990772 (+ 782us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:06.990979 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:06.991933 (+ 954us) server_negotiation.cc:299] Negotiation successful
0806 01:14:06.992243 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":281,"threads_started":1}
I20260806 01:14:06.994736 32254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:06.968710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:46091 (local address 127.25.236.129:40521)
0806 01:14:06.969756 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:06.969822 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:06.972200 (+ 2378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:06.973142 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:06.973151 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:06.973735 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:06.974482 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:06.974506 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:06.975668 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:06.975673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:06.976996 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:06.977015 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:06.977891 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:06.977900 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:06.978158 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:06.979023 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:06.979056 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:06.982522 (+ 3466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:06.987906 (+ 5384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:06.987915 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:06.987929 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:06.988774 (+ 845us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:06.989623 (+ 849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:06.989627 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:06.989629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:06.989842 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:06.990967 (+ 1125us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:06.990974 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:06.991425 (+ 451us) client_negotiation.cc:769] Sending connection context
0806 01:14:06.991757 (+ 332us) client_negotiation.cc:240] Negotiation successful
0806 01:14:06.992552 (+ 795us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":468,"threads_started":1}
I20260806 01:14:06.997210 32251 heartbeater.cc:347] Connected to a master server at 127.25.236.190:46091
I20260806 01:14:06.997545 32251 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:06.998395 32251 heartbeater.cc:511] Master 127.25.236.190:46091 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
I20260806 01:14:07.000680 31965 ts_manager.cc:194] Registered new tserver with Master: a267c4d1c23a440abf9fffcb3c72a368 (127.25.236.129:35489)
I20260806 01:14:07.002341 31965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:40521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:14:07.038021 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:46091
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:33359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:07.401062 32257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:07.401401 32257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:07.401504 32257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:07.409960 32257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:07.410094 32257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:07.410219 32257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:14:07.423066 32257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:33359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:46091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:07.424433 32257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:07.425899 32257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:07.440939 32270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:07.440927 32264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:07.441642 32268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:07.442989 32257 server_base.cc:1034] running on GCE node
I20260806 01:14:07.444085 32257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:07.445318 32257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:07.446566 32257 hybrid_clock.cc:648] HybridClock initialized: now 1785978847446520 us; error 32 us; skew 500 ppm
I20260806 01:14:07.446919 32257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:07 dist-test-slave-7nm7 krb5kdc[31894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978847, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:07.452082 32257 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:14:07.454098 32257 webserver.cc:460] Webserver started at http://127.25.236.130:37637/ using document root <none> and password file <none>
I20260806 01:14:07.454893 32257 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:07.455063 32257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:07.455402 32257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:07.458544 32257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "63568473ea8b4fc3886b05adc40a4024"
format_stamp: "Formatted at 2026-08-06 01:14:07 on dist-test-slave-7nm7"
server_key: "df7964bf982a72eb00faea924e2c6082"
server_key_iv: "7dbebc269e78036a25bab6a20885611a"
server_key_version: "encryptionkey@0"
I20260806 01:14:07.459375 32257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "63568473ea8b4fc3886b05adc40a4024"
format_stamp: "Formatted at 2026-08-06 01:14:07 on dist-test-slave-7nm7"
server_key: "df7964bf982a72eb00faea924e2c6082"
server_key_iv: "7dbebc269e78036a25bab6a20885611a"
server_key_version: "encryptionkey@0"
I20260806 01:14:07.464819 32257 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:14:07.468927 32281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:07.470376 32257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:14:07.470564 32257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "63568473ea8b4fc3886b05adc40a4024"
format_stamp: "Formatted at 2026-08-06 01:14:07 on dist-test-slave-7nm7"
server_key: "df7964bf982a72eb00faea924e2c6082"
server_key_iv: "7dbebc269e78036a25bab6a20885611a"
server_key_version: "encryptionkey@0"
I20260806 01:14:07.470783 32257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:07.504909 32257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:07.510535 32257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:07.510875 32257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:07.511926 32257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:07.513839 32257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:07.513947 32257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:07.514118 32257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:07.514195 32257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:07.555095 32257 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:38481
I20260806 01:14:07.555197 32460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:38481 every 8 connection(s)
I20260806 01:14:07.556411 32257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:14:07.565110 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 32257
I20260806 01:14:07.565330 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:14:07.565809 26546 external_mini_cluster.cc:1428] Setting key f5534e95b20058c12ad0c0b864064aa8
Aug 06 01:14:07 dist-test-slave-7nm7 krb5kdc[31894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978847, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:07.579197 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:07.559120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:45827 (local address 127.25.236.190:46091)
0806 01:14:07.559611 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:07.559616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:07.560459 (+ 843us) server_negotiation.cc:407] Connection header received
0806 01:14:07.562505 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:07.562512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:07.562607 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:07.562744 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:07.564289 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:07.565376 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:07.567193 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:07.567957 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:07.568081 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:07.572354 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:07.572359 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:07.572363 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:07.572396 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:07.575579 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:07.576346 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:07.576353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:07.576361 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:07.576485 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:07.577082 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:07.577086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:07.577088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:07.577370 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:07.577492 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:07.578735 (+ 1243us) server_negotiation.cc:299] Negotiation successful
0806 01:14:07.578919 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":263,"threads_started":1}
I20260806 01:14:07.581053 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:07.559574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:46091 (local address 127.25.236.130:45827)
0806 01:14:07.560292 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:07.560340 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:07.562155 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:07.562991 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:07.562999 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:07.563541 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:07.564100 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:07.564117 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:07.565559 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:07.565563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:07.567048 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:07.567059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:07.568228 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:07.568237 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:07.568538 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:07.569259 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:07.569298 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:07.572086 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:07.575733 (+ 3647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:07.575742 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:07.575757 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:07.576208 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:07.576603 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:07.576610 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:07.576615 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:07.576917 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:07.577539 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:07.577548 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:07.578052 (+ 504us) client_negotiation.cc:769] Sending connection context
0806 01:14:07.578713 (+ 661us) client_negotiation.cc:240] Negotiation successful
0806 01:14:07.579527 (+ 814us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":296,"threads_started":1}
I20260806 01:14:07.582832 32461 heartbeater.cc:347] Connected to a master server at 127.25.236.190:46091
I20260806 01:14:07.583254 32461 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:07.584335 32461 heartbeater.cc:511] Master 127.25.236.190:46091 requested a full tablet report, sending...
I20260806 01:14:07.586031 31965 ts_manager.cc:194] Registered new tserver with Master: 63568473ea8b4fc3886b05adc40a4024 (127.25.236.130:38481)
I20260806 01:14:07.586961 31965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:45827
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:14:07.623941 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:46091
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:33359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:07.983706 32473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:07.983990 32473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:07.984154 32473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:07.992194 32473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:07.992362 32473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:07.992540 32473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:14:08.002774 32473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:33359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:46091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:08.004159 32473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:08.005569 32473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 01:14:08.006428 32251 heartbeater.cc:503] Master 127.25.236.190:46091 was elected leader, sending a full tablet report...
W20260806 01:14:08.020440 32482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:08.020776 32483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:08.020905 32485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:08.021715 32473 server_base.cc:1034] running on GCE node
I20260806 01:14:08.022487 32473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:08.023559 32473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:08.024807 32473 hybrid_clock.cc:648] HybridClock initialized: now 1785978848024761 us; error 38 us; skew 500 ppm
I20260806 01:14:08.025184 32473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:08 dist-test-slave-7nm7 krb5kdc[31894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978848, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:08.028721 32473 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:14:08.030587 32473 webserver.cc:460] Webserver started at http://127.25.236.131:38041/ using document root <none> and password file <none>
I20260806 01:14:08.031419 32473 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:08.031554 32473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:08.031945 32473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:08.036927 32473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "20e85474e4fd4ba3ab50710426e88f49"
format_stamp: "Formatted at 2026-08-06 01:14:08 on dist-test-slave-7nm7"
server_key: "7fb7b231c9229af2a3919ed5a2907d7d"
server_key_iv: "2c959d4b78b66f1acec2a7f864e2cfca"
server_key_version: "encryptionkey@0"
I20260806 01:14:08.037895 32473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "20e85474e4fd4ba3ab50710426e88f49"
format_stamp: "Formatted at 2026-08-06 01:14:08 on dist-test-slave-7nm7"
server_key: "7fb7b231c9229af2a3919ed5a2907d7d"
server_key_iv: "2c959d4b78b66f1acec2a7f864e2cfca"
server_key_version: "encryptionkey@0"
I20260806 01:14:08.043972 32473 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260806 01:14:08.048826 32500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:08.051097 32473 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 01:14:08.051342 32473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "20e85474e4fd4ba3ab50710426e88f49"
format_stamp: "Formatted at 2026-08-06 01:14:08 on dist-test-slave-7nm7"
server_key: "7fb7b231c9229af2a3919ed5a2907d7d"
server_key_iv: "2c959d4b78b66f1acec2a7f864e2cfca"
server_key_version: "encryptionkey@0"
I20260806 01:14:08.051608 32473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:08.078094 32473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:08.084137 32473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:08.084410 32473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:08.085629 32473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:08.088467 32473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:08.088590 32473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:08.088709 32473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:08.088762 32473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:08.135874 32473 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:41369
I20260806 01:14:08.135964 32654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:41369 every 8 connection(s)
I20260806 01:14:08.137295 32473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:14:08.140944 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 32473
I20260806 01:14:08.141384 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:14:08.142524 26546 external_mini_cluster.cc:1428] Setting key 559d981be308b0d889bbb4ff88ba5757
Aug 06 01:14:08 dist-test-slave-7nm7 krb5kdc[31894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978848, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:08.163462 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:08.140588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:40759 (local address 127.25.236.190:46091)
0806 01:14:08.141177 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:08.141181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:08.142796 (+ 1615us) server_negotiation.cc:407] Connection header received
0806 01:14:08.145155 (+ 2359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:08.145159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:08.145250 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:08.145356 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:08.147242 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.148070 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.150562 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.150963 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.151077 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:08.155756 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:08.155761 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:08.155764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:08.155798 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:08.159749 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:08.160660 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:08.160668 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:08.160670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:08.160935 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:08.161455 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:08.161460 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:08.161462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:08.161790 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:08.161922 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:08.162984 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0806 01:14:08.163184 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":300,"threads_started":1}
I20260806 01:14:08.165552 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:08.141220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:46091 (local address 127.25.236.131:40759)
0806 01:14:08.142562 (+ 1342us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:08.142642 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:08.144772 (+ 2130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:08.145636 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:08.145648 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:08.146448 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:08.147047 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:08.147067 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.148384 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.148390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:08.150338 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:08.150374 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.151369 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.151376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:08.151644 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:08.152675 (+ 1031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:08.152704 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:08.155456 (+ 2752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:08.159919 (+ 4463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:08.159938 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:08.159953 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:08.160412 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:08.161073 (+ 661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:08.161076 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:08.161078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:08.161286 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:08.161971 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:08.161979 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:08.162514 (+ 535us) client_negotiation.cc:769] Sending connection context
0806 01:14:08.162816 (+ 302us) client_negotiation.cc:240] Negotiation successful
0806 01:14:08.163718 (+ 902us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1006,"thread_start_us":630,"threads_started":1}
I20260806 01:14:08.167455 32655 heartbeater.cc:347] Connected to a master server at 127.25.236.190:46091
I20260806 01:14:08.168015 32655 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:08.169216 32655 heartbeater.cc:511] Master 127.25.236.190:46091 requested a full tablet report, sending...
I20260806 01:14:08.171607 31964 ts_manager.cc:194] Registered new tserver with Master: 20e85474e4fd4ba3ab50710426e88f49 (127.25.236.131:41369)
I20260806 01:14:08.172632 31964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:40759
I20260806 01:14:08.185509 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:14:08 dist-test-slave-7nm7 krb5kdc[31894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:14:08 dist-test-slave-7nm7 krb5kdc[31894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:14:08.252806 32672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:08.234806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48496 (local address 127.25.236.129:35489)
0806 01:14:08.235752 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:08.235759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:08.235786 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:14:08.235850 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:08.235856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:08.236077 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:08.236218 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:08.237746 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.239252 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.240579 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.241182 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.241289 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:08.246606 (+ 5317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:08.246618 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:08.246643 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:08.246686 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:08.249707 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:08.250692 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:08.250699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:08.250701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:08.250838 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:08.251240 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:08.251244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:08.251246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:08.251615 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:08.251851 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:08.252179 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 01:14:08.252391 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":670,"threads_started":1}
Aug 06 01:14:08 dist-test-slave-7nm7 krb5kdc[31894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:08.272768 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:08.257793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.25.236.190:46091)
0806 01:14:08.258004 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:08.258009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:08.258081 (+ 72us) server_negotiation.cc:407] Connection header received
0806 01:14:08.258319 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:08.258329 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:08.258416 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:08.258507 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:08.259880 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.260640 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.261973 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.262487 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.262589 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:08.266854 (+ 4265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:08.266860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:08.266865 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:08.266905 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:08.269607 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:08.270522 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:08.270527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:08.270529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:08.270673 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:08.271034 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:08.271037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:08.271039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:08.271413 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:08.271602 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:08.271999 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 01:14:08.272304 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260806 01:14:08.273535 31964 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37662
I20260806 01:14:08.274727 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 32063
I20260806 01:14:08.288864 32672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:08.275900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48508 (local address 127.25.236.129:35489)
0806 01:14:08.276277 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:08.276285 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:08.276404 (+ 119us) server_negotiation.cc:407] Connection header received
0806 01:14:08.276618 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:08.276623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:08.276730 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:08.276843 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:08.278064 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.278923 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.280148 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.280550 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.280681 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:08.283308 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:08.283313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:08.283316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:08.283347 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:08.286142 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:08.286977 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:08.286983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:08.286987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:08.287131 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:08.287503 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:08.287509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:08.287512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:08.287764 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:08.288012 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:08.288318 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 01:14:08.288538 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:14:08.289676 32245 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:08.377190 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 32063
I20260806 01:14:08.404249 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 32257
Aug 06 01:14:08 dist-test-slave-7nm7 krb5kdc[31894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:14:08.422691 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:08.405311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34798 (local address 127.25.236.130:38481)
0806 01:14:08.405935 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:08.405943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:08.405969 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:14:08.406035 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:08.406041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:08.406334 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:08.406474 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:08.407834 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.408848 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.410285 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.410915 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.411015 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:08.416518 (+ 5503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:08.416531 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:08.416547 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:08.416592 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:08.419816 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:08.420515 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:08.420523 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:08.420525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:08.420644 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:08.421018 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:08.421022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:08.421024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:08.421391 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:08.421613 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:08.422006 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 01:14:08.422321 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":365,"threads_started":1}
I20260806 01:14:08.424050 32453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:08.513149 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 32257
I20260806 01:14:08.541025 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 32473
Aug 06 01:14:08 dist-test-slave-7nm7 krb5kdc[31894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:14:08.558795 32686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:08.542120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34650 (local address 127.25.236.131:41369)
0806 01:14:08.542803 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:08.542810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:08.542838 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:14:08.542900 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:08.542906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:08.543114 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:08.543271 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:08.544765 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.545845 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.547456 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:08.548102 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:08.548221 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:08.552833 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:08.552845 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:08.552861 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:08.552900 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:08.555959 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:08.556624 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:08.556631 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:08.556633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:08.556760 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:08.557183 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:08.557187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:08.557189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:08.557595 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:08.557832 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:08.558087 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 01:14:08.558483 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":377,"threads_started":1}
I20260806 01:14:08.560088 32648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:08.649389 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 32473
I20260806 01:14:08.676695 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 31914
I20260806 01:14:08.678144 32013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:08.788475 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 31914
2026-08-06T01:14:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4606 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:14:08 dist-test-slave-7nm7 krb5kdc[32692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:14:08 dist-test-slave-7nm7 krb5kdc[32692](info): set up 2 sockets
Aug 06 01:14:08 dist-test-slave-7nm7 krb5kdc[32692](info): commencing operation
krb5kdc: starting...
W20260806 01:14:10.152463 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:14:10 dist-test-slave-7nm7 krb5kdc[32692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:14:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:14:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:14:10.437731 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:36015
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:36015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:10.805527 32716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:10.805917 32716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:10.806097 32716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:10.814635 32716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:10.815027 32716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:10.815275 32716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:10.815415 32716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:10.815495 32716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:14:10.826594 32716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:36015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:36015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:10.828068 32716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:10.829459 32716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:10.845050 32726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:10.845091 32727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:10.845135 32730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:10.846132 32716 server_base.cc:1034] running on GCE node
I20260806 01:14:10.847159 32716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:10.848742 32716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:10.850018 32716 hybrid_clock.cc:648] HybridClock initialized: now 1785978850849971 us; error 65 us; skew 500 ppm
I20260806 01:14:10.850841 32716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:10 dist-test-slave-7nm7 krb5kdc[32692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978850, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:10.854539 32716 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:14:10.855813 32716 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:10.855906 32716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:10.856189 32716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:10.859295 32716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "bd33708cc08f4f2e85b1ed663f385302"
format_stamp: "Formatted at 2026-08-06 01:14:10 on dist-test-slave-7nm7"
server_key: "06ed2d69a5e5b8a8b2a928bdc844db87"
server_key_iv: "727787681bfb57514aab915c48c52346"
server_key_version: "encryptionkey@0"
I20260806 01:14:10.860051 32716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "bd33708cc08f4f2e85b1ed663f385302"
format_stamp: "Formatted at 2026-08-06 01:14:10 on dist-test-slave-7nm7"
server_key: "06ed2d69a5e5b8a8b2a928bdc844db87"
server_key_iv: "727787681bfb57514aab915c48c52346"
server_key_version: "encryptionkey@0"
I20260806 01:14:10.865579 32716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:14:10.869460 32737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:10.870975 32716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:14:10.871163 32716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "bd33708cc08f4f2e85b1ed663f385302"
format_stamp: "Formatted at 2026-08-06 01:14:10 on dist-test-slave-7nm7"
server_key: "06ed2d69a5e5b8a8b2a928bdc844db87"
server_key_iv: "727787681bfb57514aab915c48c52346"
server_key_version: "encryptionkey@0"
I20260806 01:14:10.871409 32716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:10.891762 32716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:10.896469 32716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:10.896781 32716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:10.918694 32716 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:36015
I20260806 01:14:10.918978 344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:36015 every 8 connection(s)
I20260806 01:14:10.919790 32716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:14:10.924559 345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:10.928397 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 32716
I20260806 01:14:10.928656 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:14:10.929630 26546 external_mini_cluster.cc:1428] Setting key 2cc707438fcf928298830297e26ef1ad
I20260806 01:14:10.940109 345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302: Bootstrap starting.
I20260806 01:14:10.945432 345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:10.946789 345 log.cc:824] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:10.951368 345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302: No bootstrap required, opened a new log
Aug 06 01:14:10 dist-test-slave-7nm7 krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:10.961907 345 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd33708cc08f4f2e85b1ed663f385302" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36015 } }
I20260806 01:14:10.962775 345 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:10.962917 345 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd33708cc08f4f2e85b1ed663f385302, State: Initialized, Role: FOLLOWER
I20260806 01:14:10.963099 350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:10.932524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36088 (local address 127.25.236.190:36015)
0806 01:14:10.933770 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:10.933789 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:10.934617 (+ 828us) server_negotiation.cc:407] Connection header received
0806 01:14:10.937566 (+ 2949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:10.937611 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:10.938553 (+ 942us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:10.939018 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:10.942110 (+ 3092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:10.943919 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:10.946038 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:10.947064 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:10.947231 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:10.953103 (+ 5872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:10.953112 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:10.953129 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:10.953175 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:10.956830 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:10.957801 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:10.957810 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:10.957817 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:10.958023 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:10.958526 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:10.958531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:10.958536 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:10.959422 (+ 886us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:10.960024 (+ 602us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:10.960985 (+ 961us) server_negotiation.cc:299] Negotiation successful
0806 01:14:10.961650 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":960,"thread_start_us":724,"threads_started":1}
I20260806 01:14:10.963645 345 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd33708cc08f4f2e85b1ed663f385302" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36015 } }
I20260806 01:14:10.963923 345 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:14:10.964090 345 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:14:10.964296 345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:10.966997 345 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd33708cc08f4f2e85b1ed663f385302" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36015 } }
I20260806 01:14:10.967511 345 leader_election.cc:302] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd33708cc08f4f2e85b1ed663f385302; no voters:
I20260806 01:14:10.968179 345 leader_election.cc:288] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:14:10.968423 353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:10.969105 353 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [term 1 LEADER]: Becoming Leader. State: Replica: bd33708cc08f4f2e85b1ed663f385302, State: Running, Role: LEADER
I20260806 01:14:10.969620 353 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd33708cc08f4f2e85b1ed663f385302" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36015 } }
I20260806 01:14:10.970449 345 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:10.972347 354 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd33708cc08f4f2e85b1ed663f385302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd33708cc08f4f2e85b1ed663f385302" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36015 } } }
I20260806 01:14:10.972836 354 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:10.973435 359 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd33708cc08f4f2e85b1ed663f385302. Latest consensus state: current_term: 1 leader_uuid: "bd33708cc08f4f2e85b1ed663f385302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd33708cc08f4f2e85b1ed663f385302" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36015 } } }
I20260806 01:14:10.973961 359 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:10.988327 362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:10.994397 362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:11.003571 362 catalog_manager.cc:1347] Generated new cluster ID: db04ffd31dad4f9aa740e53bae9332de
I20260806 01:14:11.003715 362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:11.012892 362 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:14:11.014510 362 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:11.026582 362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302: Generated new TSK 0
I20260806 01:14:11.027520 362 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:14:11.097743 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:36015
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:11.467149 393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:11.467424 393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:11.467535 393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:11.475270 393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:11.475379 393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:11.475550 393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:14:11.486032 393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:36015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:11.487476 393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:11.488860 393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:11.502679 401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:11.502794 402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:11.503265 404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:11.503660 393 server_base.cc:1034] running on GCE node
I20260806 01:14:11.504307 393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:11.505295 393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:11.506505 393 hybrid_clock.cc:648] HybridClock initialized: now 1785978851506439 us; error 51 us; skew 500 ppm
I20260806 01:14:11.506843 393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:11 dist-test-slave-7nm7 krb5kdc[32692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978851, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:11.510354 393 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:14:11.511639 393 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:11.511787 393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:11.512133 393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:11.516153 393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "3f3b8075adb84fe7a86ba574fc85dc5a"
format_stamp: "Formatted at 2026-08-06 01:14:11 on dist-test-slave-7nm7"
server_key: "2f2aa7f261d405e73dc777e452c8b096"
server_key_iv: "8f38bb4b68ef66fcf3acb0a9b2bcbfec"
server_key_version: "encryptionkey@0"
I20260806 01:14:11.517367 393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "3f3b8075adb84fe7a86ba574fc85dc5a"
format_stamp: "Formatted at 2026-08-06 01:14:11 on dist-test-slave-7nm7"
server_key: "2f2aa7f261d405e73dc777e452c8b096"
server_key_iv: "8f38bb4b68ef66fcf3acb0a9b2bcbfec"
server_key_version: "encryptionkey@0"
I20260806 01:14:11.523088 393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 01:14:11.526916 414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:11.528404 393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:14:11.528602 393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "3f3b8075adb84fe7a86ba574fc85dc5a"
format_stamp: "Formatted at 2026-08-06 01:14:11 on dist-test-slave-7nm7"
server_key: "2f2aa7f261d405e73dc777e452c8b096"
server_key_iv: "8f38bb4b68ef66fcf3acb0a9b2bcbfec"
server_key_version: "encryptionkey@0"
I20260806 01:14:11.528853 393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:11.549189 393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:11.553714 393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:11.554025 393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:11.555169 393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:11.556658 393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:11.556779 393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:11.556922 393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:11.557014 393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:11.597672 393 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:37005
I20260806 01:14:11.597735 593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:37005 every 8 connection(s)
I20260806 01:14:11.599247 393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:14:11.607702 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 393
I20260806 01:14:11.607975 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:14:11.608994 26546 external_mini_cluster.cc:1428] Setting key 05008dd84bfe2fcd17ed5dce78e29abc
Aug 06 01:14:11 dist-test-slave-7nm7 krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978851, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:11.624539 597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:11.602332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:50351 (local address 127.25.236.190:36015)
0806 01:14:11.602826 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:11.602831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:11.603891 (+ 1060us) server_negotiation.cc:407] Connection header received
0806 01:14:11.605843 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:11.605849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:11.605940 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:11.606068 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:11.607702 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:11.608572 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:11.610275 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:11.610851 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:11.610985 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:11.614981 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:11.614992 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:11.615000 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:11.615045 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:11.619484 (+ 4439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:11.621459 (+ 1975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:11.621469 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:11.621476 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:11.621664 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:11.622462 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:11.622470 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:11.622476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:11.622809 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:11.622940 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:11.624048 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0806 01:14:11.624255 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":272,"threads_started":1}
I20260806 01:14:11.626371 598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:11.602691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:36015 (local address 127.25.236.129:50351)
0806 01:14:11.603711 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:11.603752 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:11.605561 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:11.606364 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:11.606374 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:11.606929 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:11.607517 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:11.607533 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:11.608754 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:11.608759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:11.610083 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:11.610096 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:11.611097 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:11.611103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:11.611435 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:11.612155 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:11.612178 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:11.614736 (+ 2558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:11.619752 (+ 5016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:11.619765 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:11.619822 (+ 57us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:11.621189 (+ 1367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:11.621818 (+ 629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:11.621822 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:11.621825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:11.622030 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:11.622987 (+ 957us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:11.622999 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:11.623533 (+ 534us) client_negotiation.cc:769] Sending connection context
0806 01:14:11.623906 (+ 373us) client_negotiation.cc:240] Negotiation successful
0806 01:14:11.624553 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"thread_start_us":469,"threads_started":1}
I20260806 01:14:11.628247 595 heartbeater.cc:347] Connected to a master server at 127.25.236.190:36015
I20260806 01:14:11.628625 595 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:11.629348 595 heartbeater.cc:511] Master 127.25.236.190:36015 requested a full tablet report, sending...
I20260806 01:14:11.631585 32757 ts_manager.cc:194] Registered new tserver with Master: 3f3b8075adb84fe7a86ba574fc85dc5a (127.25.236.129:37005)
I20260806 01:14:11.633354 32757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:50351
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:14:11.669955 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:36015
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:12.073043 603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:12.073397 603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:12.073583 603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:12.082088 603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:12.082284 603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:12.082495 603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:14:12.092840 603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:36015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:12.094350 603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:12.095681 603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:12.109871 615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:12.109954 612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:12.109931 613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:12.111356 603 server_base.cc:1034] running on GCE node
I20260806 01:14:12.111961 603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:12.113229 603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:12.114705 603 hybrid_clock.cc:648] HybridClock initialized: now 1785978852114569 us; error 68 us; skew 500 ppm
I20260806 01:14:12.115585 603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:12 dist-test-slave-7nm7 krb5kdc[32692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978852, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:12.119314 603 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:14:12.120589 603 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:12.120685 603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:12.121068 603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:12.124614 603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "3c66624339a84857a3559caa241ce194"
format_stamp: "Formatted at 2026-08-06 01:14:12 on dist-test-slave-7nm7"
server_key: "57303ac29e2cd7dcc05f9839337f2348"
server_key_iv: "033cc96029c597230f1db256f1a513a6"
server_key_version: "encryptionkey@0"
I20260806 01:14:12.125514 603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "3c66624339a84857a3559caa241ce194"
format_stamp: "Formatted at 2026-08-06 01:14:12 on dist-test-slave-7nm7"
server_key: "57303ac29e2cd7dcc05f9839337f2348"
server_key_iv: "033cc96029c597230f1db256f1a513a6"
server_key_version: "encryptionkey@0"
I20260806 01:14:12.131105 603 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:14:12.136067 625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:12.137636 603 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:14:12.137833 603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "3c66624339a84857a3559caa241ce194"
format_stamp: "Formatted at 2026-08-06 01:14:12 on dist-test-slave-7nm7"
server_key: "57303ac29e2cd7dcc05f9839337f2348"
server_key_iv: "033cc96029c597230f1db256f1a513a6"
server_key_version: "encryptionkey@0"
I20260806 01:14:12.138146 603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:12.158074 603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:12.163555 603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:12.163877 603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:12.164971 603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:12.166554 603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:12.166690 603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:12.166819 603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:12.166926 603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:12.209542 603 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:37727
I20260806 01:14:12.209636 799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:37727 every 8 connection(s)
I20260806 01:14:12.210546 603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:14:12.211972 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 603
I20260806 01:14:12.212224 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:14:12.212854 26546 external_mini_cluster.cc:1428] Setting key 7d1a10e8b406fdf6ea75b21319550962
Aug 06 01:14:12 dist-test-slave-7nm7 krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978852, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:12.234956 803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:12.213218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:41099 (local address 127.25.236.190:36015)
0806 01:14:12.213748 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:12.213754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:12.214943 (+ 1189us) server_negotiation.cc:407] Connection header received
0806 01:14:12.217590 (+ 2647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:12.217594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:12.217657 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:12.217752 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:12.219882 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.220767 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.223537 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.224005 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.224121 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:12.228498 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:12.228504 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:12.228507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:12.228542 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:12.231260 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:12.232050 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:12.232057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:12.232066 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:12.232194 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:12.232643 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:12.232647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:12.232649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:12.233046 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:12.233359 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:12.234319 (+ 960us) server_negotiation.cc:299] Negotiation successful
0806 01:14:12.234569 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":254,"threads_started":1}
I20260806 01:14:12.235991 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:12.213765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:36015 (local address 127.25.236.130:41099)
0806 01:14:12.214638 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:12.214684 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:12.217292 (+ 2608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:12.218011 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:12.218025 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:12.218864 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:12.219692 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:12.219718 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.220933 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.220942 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:12.223319 (+ 2377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:12.223337 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.224258 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.224267 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:12.224576 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:12.225830 (+ 1254us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:12.225855 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:12.228228 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:12.231424 (+ 3196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:12.231434 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:12.231452 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:12.231917 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:12.232310 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:12.232315 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:12.232318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:12.232509 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:12.233263 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:12.233340 (+ 77us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:12.233738 (+ 398us) client_negotiation.cc:769] Sending connection context
0806 01:14:12.234188 (+ 450us) client_negotiation.cc:240] Negotiation successful
0806 01:14:12.234829 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":475,"threads_started":1}
I20260806 01:14:12.237715 800 heartbeater.cc:347] Connected to a master server at 127.25.236.190:36015
I20260806 01:14:12.238056 800 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:12.238826 800 heartbeater.cc:511] Master 127.25.236.190:36015 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
I20260806 01:14:12.240881 32757 ts_manager.cc:194] Registered new tserver with Master: 3c66624339a84857a3559caa241ce194 (127.25.236.130:37727)
I20260806 01:14:12.242460 32757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:41099
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:14:12.279798 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:36015
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:14:12.637781 595 heartbeater.cc:503] Master 127.25.236.190:36015 was elected leader, sending a full tablet report...
W20260806 01:14:12.677562 809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:12.677999 809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:12.678370 809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:12.687366 809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:12.687492 809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:12.687695 809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:14:12.698652 809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:36015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:12.700255 809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:12.701627 809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:12.716564 819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:12.716634 824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:12.716564 818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:12.717777 809 server_base.cc:1034] running on GCE node
I20260806 01:14:12.719023 809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:12.720258 809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:12.721491 809 hybrid_clock.cc:648] HybridClock initialized: now 1785978852721432 us; error 49 us; skew 500 ppm
I20260806 01:14:12.721894 809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:12 dist-test-slave-7nm7 krb5kdc[32692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978852, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:12.725769 809 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:14:12.727162 809 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:12.727257 809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:12.727638 809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:12.730770 809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "577fb54352fb4f2fa861b6ec952850fc"
format_stamp: "Formatted at 2026-08-06 01:14:12 on dist-test-slave-7nm7"
server_key: "b8b02732d3d1dbe57ae1e1a4bbd4ec86"
server_key_iv: "5dc9f973e8a438bb6cdeec27b2f564d8"
server_key_version: "encryptionkey@0"
I20260806 01:14:12.731602 809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "577fb54352fb4f2fa861b6ec952850fc"
format_stamp: "Formatted at 2026-08-06 01:14:12 on dist-test-slave-7nm7"
server_key: "b8b02732d3d1dbe57ae1e1a4bbd4ec86"
server_key_iv: "5dc9f973e8a438bb6cdeec27b2f564d8"
server_key_version: "encryptionkey@0"
I20260806 01:14:12.738431 809 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 01:14:12.742520 832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:12.743996 809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:14:12.744205 809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "577fb54352fb4f2fa861b6ec952850fc"
format_stamp: "Formatted at 2026-08-06 01:14:12 on dist-test-slave-7nm7"
server_key: "b8b02732d3d1dbe57ae1e1a4bbd4ec86"
server_key_iv: "5dc9f973e8a438bb6cdeec27b2f564d8"
server_key_version: "encryptionkey@0"
I20260806 01:14:12.744443 809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:12.769541 809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:12.774600 809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:12.774987 809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:12.776594 809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:12.778813 809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:12.778971 809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:12.779111 809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:12.779224 809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:12.819790 809 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:34139
I20260806 01:14:12.819885 992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:34139 every 8 connection(s)
I20260806 01:14:12.820745 809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:14:12.821832 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 809
I20260806 01:14:12.822050 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:14:12.822628 26546 external_mini_cluster.cc:1428] Setting key 929a0d18f9fbf1cf50cbcb8e91fec6ac
Aug 06 01:14:12 dist-test-slave-7nm7 krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978852, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:12.844280 1000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:12.824045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:57113 (local address 127.25.236.190:36015)
0806 01:14:12.824604 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:12.824610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:12.825624 (+ 1014us) server_negotiation.cc:407] Connection header received
0806 01:14:12.827747 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:12.827752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:12.827848 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:12.827965 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:12.829535 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.830466 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.832073 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.832531 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.832665 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:12.837212 (+ 4547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:12.837219 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:12.837223 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:12.837262 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:12.840669 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:12.841498 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:12.841502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:12.841504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:12.841685 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:12.842202 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:12.842206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:12.842208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:12.842597 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:12.842725 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:12.843603 (+ 878us) server_negotiation.cc:299] Negotiation successful
0806 01:14:12.843883 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":255,"threads_started":1}
I20260806 01:14:12.845487 1001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:12.824513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:36015 (local address 127.25.236.131:57113)
0806 01:14:12.825452 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:12.825513 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:12.827478 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:12.828201 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:12.828209 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:12.828756 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:12.829351 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:12.829368 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.830632 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.830636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:12.831881 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:12.831893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.832775 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.832783 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:12.833018 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:12.833938 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:12.833962 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:12.836881 (+ 2919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:12.840833 (+ 3952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:12.840841 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:12.840857 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:12.841371 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:12.841794 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:12.841797 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:12.841799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:12.842051 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:12.842756 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:12.842763 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:12.843199 (+ 436us) client_negotiation.cc:769] Sending connection context
0806 01:14:12.843497 (+ 298us) client_negotiation.cc:240] Negotiation successful
0806 01:14:12.844104 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":398,"threads_started":1}
I20260806 01:14:12.847236 993 heartbeater.cc:347] Connected to a master server at 127.25.236.190:36015
I20260806 01:14:12.847616 993 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:12.848482 993 heartbeater.cc:511] Master 127.25.236.190:36015 requested a full tablet report, sending...
I20260806 01:14:12.850306 32757 ts_manager.cc:194] Registered new tserver with Master: 577fb54352fb4f2fa861b6ec952850fc (127.25.236.131:34139)
I20260806 01:14:12.851773 32757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:57113
I20260806 01:14:12.852893 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:14:12.872366 1000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:12.857922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36090 (local address 127.25.236.190:36015)
0806 01:14:12.858197 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:12.858207 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:12.858725 (+ 518us) server_negotiation.cc:407] Connection header received
0806 01:14:12.858906 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:12.858910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:12.858977 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:12.859068 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:12.860571 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.861361 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.863147 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.863592 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.863697 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:12.866380 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:12.866385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:12.866388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:12.866424 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:12.869107 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:12.869843 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:12.869847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:12.869849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:12.869963 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:12.870474 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:12.870478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:12.870481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:12.870846 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:12.871036 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:12.871814 (+ 778us) server_negotiation.cc:299] Negotiation successful
0806 01:14:12.872115 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:14:12.877336 32756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:14:12.884958 32756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:14:12.928574 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:12.901465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49964 (local address 127.25.236.130:37727)
0806 01:14:12.916686 (+ 15221us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:12.916698 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:12.916740 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:14:12.917346 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:12.917360 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:12.917735 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:12.919074 (+ 1339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:12.920111 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.921616 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.924909 (+ 3293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.926514 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.926706 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:12.926900 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:12.928050 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0806 01:14:12.928260 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14992,"thread_start_us":15286,"threads_started":1}
I20260806 01:14:12.929546 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:12.900081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:37005 (local address 127.0.0.1:36124)
0806 01:14:12.911677 (+ 11596us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:12.911762 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:12.912088 (+ 326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:12.918681 (+ 6593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:12.918695 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:12.918759 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:12.920889 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:12.920901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.922837 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.922843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:12.925137 (+ 2294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:12.925152 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.928103 (+ 2951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.928112 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:12.928384 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:12.928428 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 01:14:12.928595 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 01:14:12.928778 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11181,"thread_start_us":19188,"threads_started":1}
I20260806 01:14:12.929806 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:12.900936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36124 (local address 127.25.236.129:37005)
0806 01:14:12.917630 (+ 16694us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:12.917642 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:12.917681 (+ 39us) server_negotiation.cc:407] Connection header received
0806 01:14:12.917825 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:12.917836 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:12.918123 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:12.918957 (+ 834us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:12.921079 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.922641 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.925327 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.926812 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.927023 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:12.927176 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:12.929052 (+ 1876us) server_negotiation.cc:299] Negotiation successful
0806 01:14:12.929304 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16458,"thread_start_us":17371,"threads_started":1}
I20260806 01:14:12.930987 1012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:12.900131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:37727 (local address 127.0.0.1:49964)
0806 01:14:12.915828 (+ 15697us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:12.915855 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:12.917094 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:12.919157 (+ 2063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:12.919164 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:12.919189 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:12.919920 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:12.919933 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.921797 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.921803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:12.924587 (+ 2784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:12.924604 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.927057 (+ 2453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.927135 (+ 78us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:12.927410 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:12.927469 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 01:14:12.927613 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 01:14:12.927832 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15520,"mutex_wait_us":6,"thread_start_us":12244,"threads_started":1}
I20260806 01:14:12.931912 1019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:12.913521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:34139 (local address 127.0.0.1:58058)
0806 01:14:12.921058 (+ 7537us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:12.921078 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:12.921196 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:12.922208 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:12.922218 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:12.922276 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:12.922770 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:12.922783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.925552 (+ 2769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.925558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:12.927845 (+ 2287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:12.927859 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.930143 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.930153 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:12.930557 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:12.930622 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 01:14:12.930715 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 01:14:12.930879 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7376,"thread_start_us":5311,"threads_started":1}
I20260806 01:14:12.932230 1020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:12.914013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.25.236.131:34139)
0806 01:14:12.920103 (+ 6090us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:12.920111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:12.921310 (+ 1199us) server_negotiation.cc:407] Connection header received
0806 01:14:12.921410 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:12.921418 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:12.921679 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:12.921872 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:12.923478 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.925093 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.928060 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:12.929557 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:12.929779 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:12.929953 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:12.931330 (+ 1377us) server_negotiation.cc:299] Negotiation successful
0806 01:14:12.931638 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5888,"thread_start_us":5691,"threads_started":1}
I20260806 01:14:12.935397 494 tablet_service.cc:1467] Processing CreateTablet for tablet feae821861ff4a39a7168a29152371cf (DEFAULT_TABLE table=test-table [id=5f8a810a676f4f2f8ff398d8a26b6e95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:12.936987 704 tablet_service.cc:1467] Processing CreateTablet for tablet feae821861ff4a39a7168a29152371cf (DEFAULT_TABLE table=test-table [id=5f8a810a676f4f2f8ff398d8a26b6e95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:12.937903 494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feae821861ff4a39a7168a29152371cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:12.937438 904 tablet_service.cc:1467] Processing CreateTablet for tablet feae821861ff4a39a7168a29152371cf (DEFAULT_TABLE table=test-table [id=5f8a810a676f4f2f8ff398d8a26b6e95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:12.938406 704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feae821861ff4a39a7168a29152371cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:12.938886 904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feae821861ff4a39a7168a29152371cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:12.954972 1021 tablet_bootstrap.cc:492] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc: Bootstrap starting.
I20260806 01:14:12.957145 1022 tablet_bootstrap.cc:492] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194: Bootstrap starting.
I20260806 01:14:12.959887 1023 tablet_bootstrap.cc:492] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a: Bootstrap starting.
I20260806 01:14:12.961380 1021 tablet_bootstrap.cc:654] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:12.961709 1022 tablet_bootstrap.cc:654] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:12.963238 1022 log.cc:824] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:12.963310 1021 log.cc:824] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:12.963855 1023 tablet_bootstrap.cc:654] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:12.965548 1023 log.cc:824] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:12.967772 1022 tablet_bootstrap.cc:492] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194: No bootstrap required, opened a new log
I20260806 01:14:12.968089 1022 ts_tablet_manager.cc:1397] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260806 01:14:12.968291 1021 tablet_bootstrap.cc:492] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc: No bootstrap required, opened a new log
I20260806 01:14:12.968586 1021 ts_tablet_manager.cc:1397] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.002s
I20260806 01:14:12.969482 1023 tablet_bootstrap.cc:492] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a: No bootstrap required, opened a new log
I20260806 01:14:12.969749 1023 ts_tablet_manager.cc:1397] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 01:14:12.977918 1021 raft_consensus.cc:348] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c66624339a84857a3559caa241ce194" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 37727 } } peers { permanent_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 37005 } } peers { permanent_uuid: "577fb54352fb4f2fa861b6ec952850fc" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 34139 } }
I20260806 01:14:12.978374 1021 raft_consensus.cc:374] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:12.978479 1021 raft_consensus.cc:729] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 577fb54352fb4f2fa861b6ec952850fc, State: Initialized, Role: FOLLOWER
I20260806 01:14:12.979192 1021 consensus_queue.cc:260] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c66624339a84857a3559caa241ce194" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 37727 } } peers { permanent_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 37005 } } peers { permanent_uuid: "577fb54352fb4f2fa861b6ec952850fc" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 34139 } }
I20260806 01:14:12.980417 993 heartbeater.cc:503] Master 127.25.236.190:36015 was elected leader, sending a full tablet report...
I20260806 01:14:12.981268 1021 ts_tablet_manager.cc:1428] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260806 01:14:12.981267 1022 raft_consensus.cc:348] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c66624339a84857a3559caa241ce194" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 37727 } } peers { permanent_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 37005 } } peers { permanent_uuid: "577fb54352fb4f2fa861b6ec952850fc" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 34139 } }
I20260806 01:14:12.981765 1022 raft_consensus.cc:374] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:12.981878 1022 raft_consensus.cc:729] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c66624339a84857a3559caa241ce194, State: Initialized, Role: FOLLOWER
I20260806 01:14:12.982595 1022 consensus_queue.cc:260] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c66624339a84857a3559caa241ce194" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 37727 } } peers { permanent_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 37005 } } peers { permanent_uuid: "577fb54352fb4f2fa861b6ec952850fc" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 34139 } }
I20260806 01:14:12.983881 1023 raft_consensus.cc:348] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c66624339a84857a3559caa241ce194" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 37727 } } peers { permanent_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 37005 } } peers { permanent_uuid: "577fb54352fb4f2fa861b6ec952850fc" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 34139 } }
I20260806 01:14:12.984269 1023 raft_consensus.cc:374] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:12.984390 1023 raft_consensus.cc:729] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f3b8075adb84fe7a86ba574fc85dc5a, State: Initialized, Role: FOLLOWER
I20260806 01:14:12.985030 1023 consensus_queue.cc:260] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c66624339a84857a3559caa241ce194" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 37727 } } peers { permanent_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 37005 } } peers { permanent_uuid: "577fb54352fb4f2fa861b6ec952850fc" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 34139 } }
I20260806 01:14:12.986202 800 heartbeater.cc:503] Master 127.25.236.190:36015 was elected leader, sending a full tablet report...
I20260806 01:14:12.986989 1022 ts_tablet_manager.cc:1428] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194: Time spent starting tablet: real 0.019s user 0.020s sys 0.000s
I20260806 01:14:12.989764 1023 ts_tablet_manager.cc:1428] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a: Time spent starting tablet: real 0.020s user 0.011s sys 0.006s
W20260806 01:14:13.072911 995 tablet.cc:2369] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:14:13.102612 596 tablet.cc:2369] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:14:13.153013 1030 raft_consensus.cc:482] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:14:13.153622 1030 raft_consensus.cc:504] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c66624339a84857a3559caa241ce194" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 37727 } } peers { permanent_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 37005 } } peers { permanent_uuid: "577fb54352fb4f2fa861b6ec952850fc" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 34139 } }
I20260806 01:14:13.156073 1030 leader_election.cc:288] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c66624339a84857a3559caa241ce194 (127.25.236.130:37727), 577fb54352fb4f2fa861b6ec952850fc (127.25.236.131:34139)
I20260806 01:14:13.165378 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:13.156593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:56449 (local address 127.25.236.130:37727)
0806 01:14:13.156878 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:13.156885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:13.157453 (+ 568us) server_negotiation.cc:407] Connection header received
0806 01:14:13.157767 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:13.157773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:13.157868 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:13.157948 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:13.158677 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.159622 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.162472 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.164087 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.164287 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:13.164387 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:13.164924 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 01:14:13.165078 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 01:14:13.165375 1020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:13.157451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:48545 (local address 127.25.236.131:34139)
0806 01:14:13.157625 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:13.157629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:13.158176 (+ 547us) server_negotiation.cc:407] Connection header received
0806 01:14:13.158460 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:13.158465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:13.158536 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:13.158681 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:13.159329 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.160152 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.162654 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.164318 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.164476 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:13.164525 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:13.164980 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 01:14:13.165118 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 01:14:13.165479 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:13.156472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:37727 (local address 127.25.236.129:56449)
0806 01:14:13.157180 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:13.157199 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:13.157561 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:13.158027 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:13.158035 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:13.158077 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:13.158490 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:13.158501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.159771 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.159775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:13.162301 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:13.162318 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.164340 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.164345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:13.164652 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:13.164694 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 01:14:13.164832 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 01:14:13.164989 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":109,"thread_start_us":371,"threads_started":1}
I20260806 01:14:13.165915 1032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:13.157317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:34139 (local address 127.25.236.129:48545)
0806 01:14:13.157971 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:13.157997 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:13.158119 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:13.158715 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:13.158718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:13.158740 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:13.159065 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:13.159072 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.160344 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.160356 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:13.162513 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:13.162523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.164543 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.164548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:13.164796 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:13.164834 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 01:14:13.164920 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 01:14:13.165016 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":27,"thread_start_us":338,"threads_started":1}
I20260806 01:14:13.166944 738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feae821861ff4a39a7168a29152371cf" candidate_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c66624339a84857a3559caa241ce194" is_pre_election: true
I20260806 01:14:13.167280 930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feae821861ff4a39a7168a29152371cf" candidate_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577fb54352fb4f2fa861b6ec952850fc" is_pre_election: true
I20260806 01:14:13.167595 738 raft_consensus.cc:2454] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f3b8075adb84fe7a86ba574fc85dc5a in term 0.
I20260806 01:14:13.167806 930 raft_consensus.cc:2454] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f3b8075adb84fe7a86ba574fc85dc5a in term 0.
I20260806 01:14:13.168358 415 leader_election.cc:302] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c66624339a84857a3559caa241ce194, 3f3b8075adb84fe7a86ba574fc85dc5a; no voters:
I20260806 01:14:13.168757 1030 raft_consensus.cc:2781] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:14:13.168931 1030 raft_consensus.cc:482] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:14:13.169062 1030 raft_consensus.cc:3037] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:13.171818 1030 raft_consensus.cc:504] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c66624339a84857a3559caa241ce194" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 37727 } } peers { permanent_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 37005 } } peers { permanent_uuid: "577fb54352fb4f2fa861b6ec952850fc" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 34139 } }
I20260806 01:14:13.172473 1030 leader_election.cc:288] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [CANDIDATE]: Term 1 election: Requested vote from peers 3c66624339a84857a3559caa241ce194 (127.25.236.130:37727), 577fb54352fb4f2fa861b6ec952850fc (127.25.236.131:34139)
I20260806 01:14:13.173143 738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feae821861ff4a39a7168a29152371cf" candidate_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c66624339a84857a3559caa241ce194"
I20260806 01:14:13.173316 738 raft_consensus.cc:3037] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:13.173164 930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feae821861ff4a39a7168a29152371cf" candidate_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577fb54352fb4f2fa861b6ec952850fc"
I20260806 01:14:13.173538 930 raft_consensus.cc:3037] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:13.175916 738 raft_consensus.cc:2454] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f3b8075adb84fe7a86ba574fc85dc5a in term 1.
I20260806 01:14:13.176030 930 raft_consensus.cc:2454] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f3b8075adb84fe7a86ba574fc85dc5a in term 1.
I20260806 01:14:13.176363 415 leader_election.cc:302] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c66624339a84857a3559caa241ce194, 3f3b8075adb84fe7a86ba574fc85dc5a; no voters:
I20260806 01:14:13.176843 1030 raft_consensus.cc:2781] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:13.177604 1030 raft_consensus.cc:686] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [term 1 LEADER]: Becoming Leader. State: Replica: 3f3b8075adb84fe7a86ba574fc85dc5a, State: Running, Role: LEADER
I20260806 01:14:13.178527 1030 consensus_queue.cc:237] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c66624339a84857a3559caa241ce194" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 37727 } } peers { permanent_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 37005 } } peers { permanent_uuid: "577fb54352fb4f2fa861b6ec952850fc" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 34139 } }
I20260806 01:14:13.184551 32757 catalog_manager.cc:5495] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f3b8075adb84fe7a86ba574fc85dc5a (127.25.236.129). New cstate: current_term: 1 leader_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c66624339a84857a3559caa241ce194" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 37727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f3b8075adb84fe7a86ba574fc85dc5a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 37005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "577fb54352fb4f2fa861b6ec952850fc" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 34139 } health_report { overall_health: UNKNOWN } } }
W20260806 01:14:13.212849 802 tablet.cc:2369] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:14:13.250017 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:13.243703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36132 (local address 127.25.236.129:37005)
0806 01:14:13.243906 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:13.243911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:13.243930 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:14:13.244095 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:13.244227 (+ 132us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:13.244533 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:13.244934 (+ 401us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:14:13.245337 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.246058 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.247482 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.247983 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.248117 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:13.248654 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:14:13.249286 (+ 632us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:14:13.249407 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:13.249605 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 01:14:13.249681 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:14:13.259385 930 raft_consensus.cc:1261] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc [term 1 FOLLOWER]: Refusing update from remote peer 3f3b8075adb84fe7a86ba574fc85dc5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:13.259385 738 raft_consensus.cc:1261] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194 [term 1 FOLLOWER]: Refusing update from remote peer 3f3b8075adb84fe7a86ba574fc85dc5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:13.260615 1033 consensus_queue.cc:1035] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c66624339a84857a3559caa241ce194" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 37727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:13.261042 1030 consensus_queue.cc:1035] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "577fb54352fb4f2fa861b6ec952850fc" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 34139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:13.274725 1039 mvcc.cc:205] Tried to move back new op lower bound from 7315369382938603520 to 7315369382624399360. Current Snapshot: MvccSnapshot[applied={T|T < 7315369382938603520}]
I20260806 01:14:13.277104 1040 mvcc.cc:205] Tried to move back new op lower bound from 7315369382938603520 to 7315369382624399360. Current Snapshot: MvccSnapshot[applied={T|T < 7315369382938603520}]
I20260806 01:14:13.300350 32756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:13.300669 32756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:13.305492 32756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd33708cc08f4f2e85b1ed663f385302: Sending DeleteTablet for 3 replicas of tablet feae821861ff4a39a7168a29152371cf
I20260806 01:14:13.306975 494 tablet_service.cc:1514] Processing DeleteTablet for tablet feae821861ff4a39a7168a29152371cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:13 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:36124
I20260806 01:14:13.307283 704 tablet_service.cc:1514] Processing DeleteTablet for tablet feae821861ff4a39a7168a29152371cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:13 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:49964
I20260806 01:14:13.307673 904 tablet_service.cc:1514] Processing DeleteTablet for tablet feae821861ff4a39a7168a29152371cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:13 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:58058
I20260806 01:14:13.314147 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 393
I20260806 01:14:13.319584 1052 tablet_replica.cc:330] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc: stopping tablet replica
I20260806 01:14:13.319806 1049 tablet_replica.cc:330] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a: stopping tablet replica
I20260806 01:14:13.320422 1049 raft_consensus.cc:2229] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:14:13.321007 1052 raft_consensus.cc:2229] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:13.321236 1049 raft_consensus.cc:2258] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:13.322043 1052 raft_consensus.cc:2258] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:13.323276 1051 tablet_replica.cc:330] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194: stopping tablet replica
I20260806 01:14:13.323942 1049 ts_tablet_manager.cc:1905] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:13.324910 1051 raft_consensus.cc:2229] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:13.327000 1052 ts_tablet_manager.cc:1905] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:13.327844 1051 raft_consensus.cc:2258] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 06 01:14:13 dist-test-slave-7nm7 krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:14:13.332423 1051 ts_tablet_manager.cc:1905] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:13.334685 1049 ts_tablet_manager.cc:1918] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:13.334880 1049 log.cc:1196] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/feae821861ff4a39a7168a29152371cf
I20260806 01:14:13.335461 1049 ts_tablet_manager.cc:1939] T feae821861ff4a39a7168a29152371cf P 3f3b8075adb84fe7a86ba574fc85dc5a: Deleting consensus metadata
I20260806 01:14:13.337456 32740 catalog_manager.cc:4841] TS 3f3b8075adb84fe7a86ba574fc85dc5a (127.25.236.129:37005): tablet feae821861ff4a39a7168a29152371cf (table test-table [id=5f8a810a676f4f2f8ff398d8a26b6e95]) successfully deleted
I20260806 01:14:13.340939 1052 ts_tablet_manager.cc:1918] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:13.341151 1052 log.cc:1196] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/feae821861ff4a39a7168a29152371cf
I20260806 01:14:13.341670 1052 ts_tablet_manager.cc:1939] T feae821861ff4a39a7168a29152371cf P 577fb54352fb4f2fa861b6ec952850fc: Deleting consensus metadata
I20260806 01:14:13.343003 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:13.316369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36144 (local address 127.25.236.129:37005)
0806 01:14:13.316806 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:13.316817 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:13.317874 (+ 1057us) server_negotiation.cc:407] Connection header received
0806 01:14:13.318115 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:13.318142 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:13.318295 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:13.318440 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:13.321585 (+ 3145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.322865 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.325176 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.325910 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.326015 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:13.332921 (+ 6906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:13.332937 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:13.332966 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:13.333023 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:13.337739 (+ 4716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:13.338958 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:13.338968 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:13.338972 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:13.339174 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:13.339673 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:13.339680 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:13.339685 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:13.340184 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:13.340494 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:13.342303 (+ 1809us) server_negotiation.cc:299] Negotiation successful
0806 01:14:13.342595 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242}
I20260806 01:14:13.344038 1051 ts_tablet_manager.cc:1918] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:13.344590 1051 log.cc:1196] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/feae821861ff4a39a7168a29152371cf
I20260806 01:14:13.345438 32738 catalog_manager.cc:4841] TS 577fb54352fb4f2fa861b6ec952850fc (127.25.236.131:34139): tablet feae821861ff4a39a7168a29152371cf (table test-table [id=5f8a810a676f4f2f8ff398d8a26b6e95]) successfully deleted
I20260806 01:14:13.346038 585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:13.346105 1051 ts_tablet_manager.cc:1939] T feae821861ff4a39a7168a29152371cf P 3c66624339a84857a3559caa241ce194: Deleting consensus metadata
I20260806 01:14:13.348196 32738 catalog_manager.cc:4841] TS 3c66624339a84857a3559caa241ce194 (127.25.236.130:37727): tablet feae821861ff4a39a7168a29152371cf (table test-table [id=5f8a810a676f4f2f8ff398d8a26b6e95]) successfully deleted
I20260806 01:14:13.454643 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 393
I20260806 01:14:13.485234 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 603
Aug 06 01:14:13 dist-test-slave-7nm7 krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:14:13.502573 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:13.486194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49980 (local address 127.25.236.130:37727)
0806 01:14:13.486409 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:13.486414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:13.486451 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:14:13.486646 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:13.486650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:13.486713 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:13.486807 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:13.488328 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.489064 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.490455 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.490869 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.490972 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:13.496462 (+ 5490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:13.496478 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:13.496494 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:13.496531 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:13.499378 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:13.500165 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:13.500173 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:13.500175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:13.500316 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:13.500768 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:13.500772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:13.500774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:13.501218 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:13.501448 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:13.501889 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 01:14:13.502072 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 01:14:13.503396 794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:13.601536 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 603
I20260806 01:14:13.633824 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 809
Aug 06 01:14:13 dist-test-slave-7nm7 krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:14:13.651778 1020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:13.635075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58066 (local address 127.25.236.131:34139)
0806 01:14:13.635494 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:13.635500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:13.635525 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:14:13.635611 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:13.635615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:13.635692 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:13.635799 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:13.637310 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.638637 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.639967 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:13.640643 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:13.640778 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:13.645475 (+ 4697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:13.645492 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:13.645509 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:13.645547 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:13.648632 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:13.649495 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:13.649503 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:13.649505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:13.649672 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:13.650048 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:13.650052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:13.650054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:13.650568 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:13.650810 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:13.651197 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 01:14:13.651378 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260806 01:14:13.652848 987 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:13.746855 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 809
I20260806 01:14:13.773780 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 32716
I20260806 01:14:13.774983 338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:13.893126 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 32716
2026-08-06T01:14:13Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (5101 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T01:14:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:14:13Z Disabled control of system clock
I20260806 01:14:13.987028 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:33463
--webserver_interface=127.25.236.190
--webserver_port=0
--builtin_ntp_servers=127.25.236.148:35107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:33463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 01:14:14.361644 1065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:14.361960 1065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:14.362056 1065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:14.369589 1065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:14.369710 1065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:14.369750 1065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:14.369800 1065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:14.369845 1065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:14:14.379098 1065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:35107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:33463
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:33463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:14.380388 1065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:14.382133 1065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:14.395191 1072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:14.395411 1073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:14.395160 1075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:14.396416 1065 server_base.cc:1034] running on GCE node
I20260806 01:14:14.397243 1065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:14.398831 1065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:14.400094 1065 hybrid_clock.cc:648] HybridClock initialized: now 1785978854400045 us; error 51 us; skew 500 ppm
I20260806 01:14:14.400522 1065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:14:14.403213 1065 webserver.cc:460] Webserver started at http://127.25.236.190:35699/ using document root <none> and password file <none>
I20260806 01:14:14.404124 1065 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:14.404264 1065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:14.404632 1065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:14.407732 1065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "8ea5aceb9800434d9ecd0d5c2d581e9a"
format_stamp: "Formatted at 2026-08-06 01:14:14 on dist-test-slave-7nm7"
server_key: "7bf0d42d082846150e4b1a83acff3e50"
server_key_iv: "9acedc2af4296baded07d3c9eb6da67b"
server_key_version: "encryptionkey@0"
I20260806 01:14:14.408556 1065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "8ea5aceb9800434d9ecd0d5c2d581e9a"
format_stamp: "Formatted at 2026-08-06 01:14:14 on dist-test-slave-7nm7"
server_key: "7bf0d42d082846150e4b1a83acff3e50"
server_key_iv: "9acedc2af4296baded07d3c9eb6da67b"
server_key_version: "encryptionkey@0"
I20260806 01:14:14.413862 1065 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:14:14.417512 1085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:14.418912 1065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:14:14.419095 1065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "8ea5aceb9800434d9ecd0d5c2d581e9a"
format_stamp: "Formatted at 2026-08-06 01:14:14 on dist-test-slave-7nm7"
server_key: "7bf0d42d082846150e4b1a83acff3e50"
server_key_iv: "9acedc2af4296baded07d3c9eb6da67b"
server_key_version: "encryptionkey@0"
I20260806 01:14:14.419322 1065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:14.425320 1065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:14.426123 1065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:14.426501 1065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:14.449620 1065 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:33463
I20260806 01:14:14.449664 1165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:33463 every 8 connection(s)
I20260806 01:14:14.451124 1065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:14:14.455446 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 1065
I20260806 01:14:14.455663 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:14:14.455791 1166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:14.456669 26546 external_mini_cluster.cc:1428] Setting key 51dafe0722026c3f246130a986d5147a
I20260806 01:14:14.466725 1171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:14.459062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46352 (local address 127.25.236.190:33463)
0806 01:14:14.460204 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:14.460215 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:14.460256 (+ 41us) server_negotiation.cc:407] Connection header received
0806 01:14:14.462630 (+ 2374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:14.462656 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:14.463261 (+ 605us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:14.463726 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:14.464960 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:14.464966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:14.464979 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:14.465003 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:14.465069 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:14.465087 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:14.465345 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:14.465487 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 01:14:14.465743 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":413,"threads_started":1}
I20260806 01:14:14.472756 1166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a: Bootstrap starting.
I20260806 01:14:14.476254 1166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:14.477272 1166 log.cc:824] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:14.480733 1166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a: No bootstrap required, opened a new log
I20260806 01:14:14.490636 1166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea5aceb9800434d9ecd0d5c2d581e9a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33463 } }
I20260806 01:14:14.491160 1166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:14.491334 1166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea5aceb9800434d9ecd0d5c2d581e9a, State: Initialized, Role: FOLLOWER
I20260806 01:14:14.491959 1166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea5aceb9800434d9ecd0d5c2d581e9a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33463 } }
I20260806 01:14:14.492228 1166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:14:14.492381 1166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:14:14.492555 1166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:14.494974 1166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea5aceb9800434d9ecd0d5c2d581e9a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33463 } }
I20260806 01:14:14.495608 1166 leader_election.cc:302] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ea5aceb9800434d9ecd0d5c2d581e9a; no voters:
I20260806 01:14:14.496281 1166 leader_election.cc:288] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:14:14.496451 1178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:14.497349 1178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [term 1 LEADER]: Becoming Leader. State: Replica: 8ea5aceb9800434d9ecd0d5c2d581e9a, State: Running, Role: LEADER
I20260806 01:14:14.497910 1178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea5aceb9800434d9ecd0d5c2d581e9a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33463 } }
I20260806 01:14:14.498641 1166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:14.500909 1179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ea5aceb9800434d9ecd0d5c2d581e9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea5aceb9800434d9ecd0d5c2d581e9a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33463 } } }
I20260806 01:14:14.501312 1179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:14.500990 1180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ea5aceb9800434d9ecd0d5c2d581e9a. Latest consensus state: current_term: 1 leader_uuid: "8ea5aceb9800434d9ecd0d5c2d581e9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea5aceb9800434d9ecd0d5c2d581e9a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33463 } } }
I20260806 01:14:14.501663 1180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:14.503002 1186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:14.508651 1186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:14.519244 1186 catalog_manager.cc:1347] Generated new cluster ID: ec6b41590160438eaab42cccf5c01048
I20260806 01:14:14.519382 1186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:14.548766 1186 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:14:14.549284 1186 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:14.563745 1186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a: Generated new TSK 0
I20260806 01:14:14.564931 1186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:14:14.573016 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33463
--builtin_ntp_servers=127.25.236.148:35107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 01:14:14.988158 1201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:14.988482 1201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:14.988646 1201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:14.997790 1201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:14.997913 1201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:14.998101 1201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:14:15.010705 1201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:35107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:15.012293 1201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:15.014430 1201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:15.030733 1215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:15.030894 1214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:15.030829 1218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:15.032492 1201 server_base.cc:1034] running on GCE node
I20260806 01:14:15.033150 1201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:15.034142 1201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:15.035418 1201 hybrid_clock.cc:648] HybridClock initialized: now 1785978855035373 us; error 33 us; skew 500 ppm
I20260806 01:14:15.035742 1201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:14:15.038442 1201 webserver.cc:460] Webserver started at http://127.25.236.129:38041/ using document root <none> and password file <none>
I20260806 01:14:15.039145 1201 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:15.039237 1201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:15.039510 1201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:15.042454 1201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "8c3cfcc74d6e44e8a9e97df726938fbb"
format_stamp: "Formatted at 2026-08-06 01:14:15 on dist-test-slave-7nm7"
server_key: "40bfa0c4467c7932c04aa76f017020de"
server_key_iv: "740ecef3a7445a54f917402c2dbf9d4e"
server_key_version: "encryptionkey@0"
I20260806 01:14:15.043252 1201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "8c3cfcc74d6e44e8a9e97df726938fbb"
format_stamp: "Formatted at 2026-08-06 01:14:15 on dist-test-slave-7nm7"
server_key: "40bfa0c4467c7932c04aa76f017020de"
server_key_iv: "740ecef3a7445a54f917402c2dbf9d4e"
server_key_version: "encryptionkey@0"
I20260806 01:14:15.049484 1201 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 01:14:15.054174 1226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:15.055838 1201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:14:15.055999 1201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "8c3cfcc74d6e44e8a9e97df726938fbb"
format_stamp: "Formatted at 2026-08-06 01:14:15 on dist-test-slave-7nm7"
server_key: "40bfa0c4467c7932c04aa76f017020de"
server_key_iv: "740ecef3a7445a54f917402c2dbf9d4e"
server_key_version: "encryptionkey@0"
I20260806 01:14:15.056212 1201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:15.062270 1201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:15.063927 1201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:15.064391 1201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:15.065773 1201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:15.068270 1201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:15.068383 1201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:15.068631 1201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:15.068742 1201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:15.112607 1201 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:34227
I20260806 01:14:15.112788 1401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:34227 every 8 connection(s)
I20260806 01:14:15.114024 1201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:14:15.115154 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 1201
I20260806 01:14:15.115356 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:14:15.116112 26546 external_mini_cluster.cc:1428] Setting key 6a958aee6c565318ea608d452b5a0af4
I20260806 01:14:15.119598 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33463
--builtin_ntp_servers=127.25.236.148:35107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 01:14:15.124878 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:15.117462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:35169 (local address 127.25.236.190:33463)
0806 01:14:15.117937 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:15.117943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:15.119260 (+ 1317us) server_negotiation.cc:407] Connection header received
0806 01:14:15.121548 (+ 2288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:15.121554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:15.121628 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:15.121783 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:15.123673 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:15.123682 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:15.123690 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:15.123716 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:15.123778 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:15.123797 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:15.123957 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:15.124377 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 01:14:15.124455 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":233,"threads_started":1}
I20260806 01:14:15.126135 1408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:15.117918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:33463 (local address 127.25.236.129:35169)
0806 01:14:15.119016 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:15.119066 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:15.121234 (+ 2168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:15.122031 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:15.122040 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:15.122302 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:15.123215 (+ 913us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:14:15.123218 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:15.123259 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:14:15.123374 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:15.123949 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:15.123959 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:15.123966 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 01:14:15.124409 (+ 443us) client_negotiation.cc:240] Negotiation successful
0806 01:14:15.124645 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":386,"threads_started":1}
I20260806 01:14:15.129741 1402 heartbeater.cc:347] Connected to a master server at 127.25.236.190:33463
I20260806 01:14:15.130347 1402 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:15.131193 1402 heartbeater.cc:511] Master 127.25.236.190:33463 requested a full tablet report, sending...
I20260806 01:14:15.134357 1111 ts_manager.cc:194] Registered new tserver with Master: 8c3cfcc74d6e44e8a9e97df726938fbb (127.25.236.129:34227)
W20260806 01:14:15.504856 1409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:15.505239 1409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:15.505424 1409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:15.514001 1409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:15.514468 1409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:15.514680 1409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:14:15.527004 1409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:35107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:15.528535 1409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:15.530575 1409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:15.545740 1417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:15.546021 1420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:15.546422 1422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:15.547571 1409 server_base.cc:1034] running on GCE node
I20260806 01:14:15.548259 1409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:15.549613 1409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:15.550976 1409 hybrid_clock.cc:648] HybridClock initialized: now 1785978855550886 us; error 81 us; skew 500 ppm
I20260806 01:14:15.551491 1409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:14:15.554721 1409 webserver.cc:460] Webserver started at http://127.25.236.130:33227/ using document root <none> and password file <none>
I20260806 01:14:15.555622 1409 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:15.555768 1409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:15.556253 1409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:15.561137 1409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "5c4f465260f94bdfac8e03636ec3d3f7"
format_stamp: "Formatted at 2026-08-06 01:14:15 on dist-test-slave-7nm7"
server_key: "bf5bbafd9f13863292191fd55ed216d1"
server_key_iv: "ee448bede23c8a7f72c67ab84e096526"
server_key_version: "encryptionkey@0"
I20260806 01:14:15.562417 1409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "5c4f465260f94bdfac8e03636ec3d3f7"
format_stamp: "Formatted at 2026-08-06 01:14:15 on dist-test-slave-7nm7"
server_key: "bf5bbafd9f13863292191fd55ed216d1"
server_key_iv: "ee448bede23c8a7f72c67ab84e096526"
server_key_version: "encryptionkey@0"
I20260806 01:14:15.569590 1409 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260806 01:14:15.574728 1431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:15.576483 1409 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 01:14:15.576694 1409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "5c4f465260f94bdfac8e03636ec3d3f7"
format_stamp: "Formatted at 2026-08-06 01:14:15 on dist-test-slave-7nm7"
server_key: "bf5bbafd9f13863292191fd55ed216d1"
server_key_iv: "ee448bede23c8a7f72c67ab84e096526"
server_key_version: "encryptionkey@0"
I20260806 01:14:15.576958 1409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:15.586453 1409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:15.587574 1409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:15.588011 1409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:15.589087 1409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:15.591063 1409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:15.591215 1409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:15.591324 1409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:15.591383 1409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:15.639727 1409 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:36671
I20260806 01:14:15.639775 1602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:36671 every 8 connection(s)
I20260806 01:14:15.641223 1409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:14:15.642604 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 1409
I20260806 01:14:15.642827 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:14:15.643392 26546 external_mini_cluster.cc:1428] Setting key 957190d7b539ac18b83335ff74f83cfb
I20260806 01:14:15.647444 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33463
--builtin_ntp_servers=127.25.236.148:35107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 01:14:15.653399 1609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:15.644674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:56491 (local address 127.25.236.190:33463)
0806 01:14:15.645346 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:15.645352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:15.646620 (+ 1268us) server_negotiation.cc:407] Connection header received
0806 01:14:15.649055 (+ 2435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:15.649062 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:15.649163 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:15.649323 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:15.651943 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:15.651955 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:15.651963 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:15.652004 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:15.652083 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:15.652208 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:15.652515 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:15.652942 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 01:14:15.653019 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":367,"threads_started":1}
I20260806 01:14:15.655103 1610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:15.645211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:33463 (local address 127.25.236.130:56491)
0806 01:14:15.646423 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:15.646481 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:15.648538 (+ 2057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:15.649683 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:15.649695 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:15.649971 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:15.651243 (+ 1272us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:14:15.651249 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:15.651352 (+ 103us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:14:15.651536 (+ 184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:15.652678 (+ 1142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:15.652692 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:15.652703 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 01:14:15.653726 (+ 1023us) client_negotiation.cc:240] Negotiation successful
0806 01:14:15.653995 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":352,"threads_started":1}
I20260806 01:14:15.657579 1604 heartbeater.cc:347] Connected to a master server at 127.25.236.190:33463
I20260806 01:14:15.658022 1604 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:15.658795 1604 heartbeater.cc:511] Master 127.25.236.190:33463 requested a full tablet report, sending...
I20260806 01:14:15.660537 1111 ts_manager.cc:194] Registered new tserver with Master: 5c4f465260f94bdfac8e03636ec3d3f7 (127.25.236.130:36671)
W20260806 01:14:16.019694 1611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:16.020000 1611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:16.020102 1611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:16.028672 1611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:16.029031 1611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:16.029203 1611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:14:16.041040 1611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:35107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:16.042428 1611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:16.044301 1611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:16.060662 1618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:16.060639 1617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:16.060806 1621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:16.062057 1611 server_base.cc:1034] running on GCE node
I20260806 01:14:16.062844 1611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:16.064477 1611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:16.065775 1611 hybrid_clock.cc:648] HybridClock initialized: now 1785978856065720 us; error 72 us; skew 500 ppm
I20260806 01:14:16.066365 1611 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:14:16.069027 1611 webserver.cc:460] Webserver started at http://127.25.236.131:45723/ using document root <none> and password file <none>
I20260806 01:14:16.070029 1611 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:16.070492 1611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:16.070950 1611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:16.074214 1611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "60badc8e4d254ee68b3668391059708a"
format_stamp: "Formatted at 2026-08-06 01:14:16 on dist-test-slave-7nm7"
server_key: "c012c71f9e6f1ce3d98df5281cf6fdae"
server_key_iv: "5c9044a0f2fde95ea57eb05682de3bb3"
server_key_version: "encryptionkey@0"
I20260806 01:14:16.075213 1611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "60badc8e4d254ee68b3668391059708a"
format_stamp: "Formatted at 2026-08-06 01:14:16 on dist-test-slave-7nm7"
server_key: "c012c71f9e6f1ce3d98df5281cf6fdae"
server_key_iv: "5c9044a0f2fde95ea57eb05682de3bb3"
server_key_version: "encryptionkey@0"
I20260806 01:14:16.081637 1611 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 01:14:16.086164 1629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:16.087779 1611 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:14:16.087980 1611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "60badc8e4d254ee68b3668391059708a"
format_stamp: "Formatted at 2026-08-06 01:14:16 on dist-test-slave-7nm7"
server_key: "c012c71f9e6f1ce3d98df5281cf6fdae"
server_key_iv: "5c9044a0f2fde95ea57eb05682de3bb3"
server_key_version: "encryptionkey@0"
I20260806 01:14:16.088251 1611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:16.095016 1611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:16.095914 1611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:16.096227 1611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:16.097273 1611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:16.099020 1611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:16.099164 1611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:16.099298 1611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:16.099393 1611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:16.138513 1402 heartbeater.cc:503] Master 127.25.236.190:33463 was elected leader, sending a full tablet report...
I20260806 01:14:16.144371 1611 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:42461
I20260806 01:14:16.144484 1800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:42461 every 8 connection(s)
I20260806 01:14:16.146721 1611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:14:16.151996 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 1611
I20260806 01:14:16.152230 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:14:16.152731 26546 external_mini_cluster.cc:1428] Setting key ea38ed35b44536c9f3a7df0236dcd784
I20260806 01:14:16.157882 1609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.150521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:47117 (local address 127.25.236.190:33463)
0806 01:14:16.150737 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:16.150742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:16.152967 (+ 2225us) server_negotiation.cc:407] Connection header received
0806 01:14:16.155317 (+ 2350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:16.155322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:16.155410 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:16.155531 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:16.157099 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:16.157104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:16.157109 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:16.157131 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:16.157173 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:16.157185 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:16.157314 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:16.157530 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 01:14:16.157614 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:14:16.158972 1804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.151486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:33463 (local address 127.25.236.131:47117)
0806 01:14:16.152763 (+ 1277us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:16.152813 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:16.155003 (+ 2190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:16.155817 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:16.155827 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:16.155928 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:16.156714 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:14:16.156718 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:16.156775 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:14:16.156899 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:16.157286 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:16.157297 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:16.157305 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 01:14:16.157613 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 01:14:16.157890 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":435,"threads_started":1}
I20260806 01:14:16.161091 1801 heartbeater.cc:347] Connected to a master server at 127.25.236.190:33463
I20260806 01:14:16.161633 1801 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:16.162675 1801 heartbeater.cc:511] Master 127.25.236.190:33463 requested a full tablet report, sending...
I20260806 01:14:16.164831 1111 ts_manager.cc:194] Registered new tserver with Master: 60badc8e4d254ee68b3668391059708a (127.25.236.131:42461)
I20260806 01:14:16.170711 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:14:16.179936 1609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.176349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46356 (local address 127.25.236.190:33463)
0806 01:14:16.176672 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:16.176679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:16.177002 (+ 323us) server_negotiation.cc:407] Connection header received
0806 01:14:16.177378 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:16.177383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:16.177445 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:16.177559 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:16.179159 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:16.179171 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:16.179174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:16.179193 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:16.179229 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:16.179243 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:16.179404 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:16.179533 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 01:14:16.179606 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260806 01:14:16.185420 1111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:14:16.193578 1111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:14:16.217672 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.210009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:36671 (local address 127.0.0.1:59046)
0806 01:14:16.213418 (+ 3409us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:16.213442 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:16.213564 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:16.214272 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:16.214285 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:16.214406 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:16.215800 (+ 1394us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:14:16.215806 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:16.215829 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:14:16.215893 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:16.216480 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:16.216497 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:16.216510 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 01:14:16.216733 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 01:14:16.216857 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3235,"thread_start_us":3032,"threads_started":1}
I20260806 01:14:16.217749 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.209901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:42461 (local address 127.0.0.1:59626)
0806 01:14:16.210847 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:16.210884 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:16.211029 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:16.214754 (+ 3725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:16.214761 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:16.214793 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:16.215649 (+ 856us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:14:16.215654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:16.215684 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:14:16.215789 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:16.217026 (+ 1237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:16.217031 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:16.217035 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 01:14:16.217196 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 01:14:16.217247 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":488,"threads_started":1}
I20260806 01:14:16.218273 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.210071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59626 (local address 127.25.236.131:42461)
0806 01:14:16.214174 (+ 4103us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:16.214182 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:16.214216 (+ 34us) server_negotiation.cc:407] Connection header received
0806 01:14:16.214332 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:16.214339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:16.214519 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:16.215189 (+ 670us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:16.216785 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:16.216794 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:16.216806 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:16.216825 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:16.216879 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:16.216898 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:16.217621 (+ 723us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:16.217697 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 01:14:16.217813 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3916,"thread_start_us":3749,"threads_started":1}
I20260806 01:14:16.218874 1826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.211495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59046 (local address 127.25.236.130:36671)
0806 01:14:16.213594 (+ 2099us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:16.213606 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:16.213639 (+ 33us) server_negotiation.cc:407] Connection header received
0806 01:14:16.213721 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:16.213733 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:16.214000 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:16.214182 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:16.216145 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:16.216159 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:16.216177 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:16.216223 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:16.216301 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:16.216329 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:16.218060 (+ 1731us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:16.218263 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 01:14:16.218384 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":316,"threads_started":1}
I20260806 01:14:16.222446 1506 tablet_service.cc:1467] Processing CreateTablet for tablet 94662fd4f14145ada538ebb6374d6737 (DEFAULT_TABLE table=test-table [id=9bb8ae2e7fb046d092eb3d106c7824ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:16.222443 1699 tablet_service.cc:1467] Processing CreateTablet for tablet 94662fd4f14145ada538ebb6374d6737 (DEFAULT_TABLE table=test-table [id=9bb8ae2e7fb046d092eb3d106c7824ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:16.223562 1829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.213997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48260 (local address 127.25.236.129:34227)
0806 01:14:16.219874 (+ 5877us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:16.219883 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:16.219913 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:14:16.219987 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:16.219999 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:16.220204 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:16.220365 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:16.222487 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:16.222496 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:16.222510 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:16.222531 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:16.222784 (+ 253us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:16.222808 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:16.222988 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:16.223079 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 01:14:16.223185 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5709,"thread_start_us":5545,"threads_started":1}
I20260806 01:14:16.223606 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.214983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34227 (local address 127.0.0.1:48260)
0806 01:14:16.215588 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:16.215606 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:16.215702 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:16.221313 (+ 5611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:16.221319 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:16.221344 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:16.222197 (+ 853us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:14:16.222200 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:16.222218 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:14:16.222291 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:16.222935 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:16.222942 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:16.222950 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 01:14:16.223087 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 01:14:16.223144 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":625,"threads_started":1}
I20260806 01:14:16.224080 1506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94662fd4f14145ada538ebb6374d6737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:16.224603 1699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94662fd4f14145ada538ebb6374d6737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:16.227306 1301 tablet_service.cc:1467] Processing CreateTablet for tablet 94662fd4f14145ada538ebb6374d6737 (DEFAULT_TABLE table=test-table [id=9bb8ae2e7fb046d092eb3d106c7824ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:16.228606 1301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94662fd4f14145ada538ebb6374d6737. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:16.239806 1831 tablet_bootstrap.cc:492] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a: Bootstrap starting.
I20260806 01:14:16.241631 1834 tablet_bootstrap.cc:492] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7: Bootstrap starting.
I20260806 01:14:16.243162 1832 tablet_bootstrap.cc:492] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb: Bootstrap starting.
I20260806 01:14:16.245757 1834 tablet_bootstrap.cc:654] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:16.246219 1831 tablet_bootstrap.cc:654] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:16.246562 1832 tablet_bootstrap.cc:654] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:16.247642 1832 log.cc:824] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:16.247633 1834 log.cc:824] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:16.247977 1831 log.cc:824] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:16.251040 1832 tablet_bootstrap.cc:492] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb: No bootstrap required, opened a new log
I20260806 01:14:16.251194 1834 tablet_bootstrap.cc:492] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7: No bootstrap required, opened a new log
I20260806 01:14:16.251322 1832 ts_tablet_manager.cc:1397] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260806 01:14:16.251351 1831 tablet_bootstrap.cc:492] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a: No bootstrap required, opened a new log
I20260806 01:14:16.251477 1834 ts_tablet_manager.cc:1397] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260806 01:14:16.251616 1831 ts_tablet_manager.cc:1397] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260806 01:14:16.258632 1832 raft_consensus.cc:348] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60badc8e4d254ee68b3668391059708a" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 42461 } } peers { permanent_uuid: "5c4f465260f94bdfac8e03636ec3d3f7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36671 } } peers { permanent_uuid: "8c3cfcc74d6e44e8a9e97df726938fbb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34227 } }
I20260806 01:14:16.259004 1832 raft_consensus.cc:374] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:16.259114 1832 raft_consensus.cc:729] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3cfcc74d6e44e8a9e97df726938fbb, State: Initialized, Role: FOLLOWER
I20260806 01:14:16.259768 1832 consensus_queue.cc:260] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60badc8e4d254ee68b3668391059708a" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 42461 } } peers { permanent_uuid: "5c4f465260f94bdfac8e03636ec3d3f7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36671 } } peers { permanent_uuid: "8c3cfcc74d6e44e8a9e97df726938fbb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34227 } }
I20260806 01:14:16.261624 1832 ts_tablet_manager.cc:1428] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260806 01:14:16.263049 1831 raft_consensus.cc:348] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60badc8e4d254ee68b3668391059708a" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 42461 } } peers { permanent_uuid: "5c4f465260f94bdfac8e03636ec3d3f7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36671 } } peers { permanent_uuid: "8c3cfcc74d6e44e8a9e97df726938fbb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34227 } }
I20260806 01:14:16.263501 1831 raft_consensus.cc:374] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:16.263619 1831 raft_consensus.cc:729] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60badc8e4d254ee68b3668391059708a, State: Initialized, Role: FOLLOWER
I20260806 01:14:16.263576 1834 raft_consensus.cc:348] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60badc8e4d254ee68b3668391059708a" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 42461 } } peers { permanent_uuid: "5c4f465260f94bdfac8e03636ec3d3f7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36671 } } peers { permanent_uuid: "8c3cfcc74d6e44e8a9e97df726938fbb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34227 } }
I20260806 01:14:16.263988 1834 raft_consensus.cc:374] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:16.264074 1834 raft_consensus.cc:729] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4f465260f94bdfac8e03636ec3d3f7, State: Initialized, Role: FOLLOWER
I20260806 01:14:16.264284 1831 consensus_queue.cc:260] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60badc8e4d254ee68b3668391059708a" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 42461 } } peers { permanent_uuid: "5c4f465260f94bdfac8e03636ec3d3f7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36671 } } peers { permanent_uuid: "8c3cfcc74d6e44e8a9e97df726938fbb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34227 } }
I20260806 01:14:16.264648 1834 consensus_queue.cc:260] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60badc8e4d254ee68b3668391059708a" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 42461 } } peers { permanent_uuid: "5c4f465260f94bdfac8e03636ec3d3f7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36671 } } peers { permanent_uuid: "8c3cfcc74d6e44e8a9e97df726938fbb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34227 } }
I20260806 01:14:16.265580 1801 heartbeater.cc:503] Master 127.25.236.190:33463 was elected leader, sending a full tablet report...
I20260806 01:14:16.266333 1831 ts_tablet_manager.cc:1428] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260806 01:14:16.267278 1604 heartbeater.cc:503] Master 127.25.236.190:33463 was elected leader, sending a full tablet report...
I20260806 01:14:16.268795 1834 ts_tablet_manager.cc:1428] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260806 01:14:16.357134 1842 raft_consensus.cc:482] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:14:16.357491 1842 raft_consensus.cc:504] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60badc8e4d254ee68b3668391059708a" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 42461 } } peers { permanent_uuid: "5c4f465260f94bdfac8e03636ec3d3f7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36671 } } peers { permanent_uuid: "8c3cfcc74d6e44e8a9e97df726938fbb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34227 } }
I20260806 01:14:16.359704 1842 leader_election.cc:288] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c4f465260f94bdfac8e03636ec3d3f7 (127.25.236.130:36671), 8c3cfcc74d6e44e8a9e97df726938fbb (127.25.236.129:34227)
I20260806 01:14:16.363286 1826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.360246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:40269 (local address 127.25.236.130:36671)
0806 01:14:16.360433 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:16.360441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:16.360468 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:14:16.360906 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:16.360914 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:16.360986 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:16.361120 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:16.362596 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:16.362610 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:16.362613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:16.362628 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:16.362666 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:16.362678 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:16.362804 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:16.362937 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 01:14:16.362998 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:14:16.363296 1804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.360058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:36671 (local address 127.25.236.131:40269)
0806 01:14:16.360262 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:16.360368 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:16.360493 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:16.361410 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:16.361416 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:16.361448 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:16.362382 (+ 934us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:14:16.362384 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:16.362400 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:14:16.362440 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:16.362801 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:16.362806 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:16.362811 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 01:14:16.362938 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 01:14:16.362995 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 01:14:16.363554 1829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.361067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:39905 (local address 127.25.236.129:34227)
0806 01:14:16.361403 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:16.361410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:16.361469 (+ 59us) server_negotiation.cc:407] Connection header received
0806 01:14:16.361622 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:16.361629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:16.361698 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:16.361813 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:16.362813 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:16.362821 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:16.362830 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:16.362846 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:16.362876 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:16.362887 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:16.363027 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:16.363094 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 01:14:16.363155 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:14:16.363741 1844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.360674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34227 (local address 127.25.236.131:39905)
0806 01:14:16.361394 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:16.361410 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:16.361496 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:16.361874 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:16.361879 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:16.361905 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:16.362610 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:14:16.362612 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:16.362625 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:14:16.362662 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:16.362992 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:16.362996 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:16.363002 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 01:14:16.363119 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 01:14:16.363167 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":408,"threads_started":1}
I20260806 01:14:16.364631 1532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94662fd4f14145ada538ebb6374d6737" candidate_uuid: "60badc8e4d254ee68b3668391059708a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4f465260f94bdfac8e03636ec3d3f7" is_pre_election: true
I20260806 01:14:16.364964 1340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94662fd4f14145ada538ebb6374d6737" candidate_uuid: "60badc8e4d254ee68b3668391059708a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3cfcc74d6e44e8a9e97df726938fbb" is_pre_election: true
I20260806 01:14:16.365353 1532 raft_consensus.cc:2454] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60badc8e4d254ee68b3668391059708a in term 0.
I20260806 01:14:16.365595 1340 raft_consensus.cc:2454] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60badc8e4d254ee68b3668391059708a in term 0.
I20260806 01:14:16.366262 1630 leader_election.cc:302] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c4f465260f94bdfac8e03636ec3d3f7, 60badc8e4d254ee68b3668391059708a; no voters:
I20260806 01:14:16.366712 1842 raft_consensus.cc:2781] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:14:16.366874 1842 raft_consensus.cc:482] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:14:16.367007 1842 raft_consensus.cc:3037] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [term 0 FOLLOWER]: Advancing to term 1
W20260806 01:14:16.367022 1404 tablet.cc:2369] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:14:16.369594 1842 raft_consensus.cc:504] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60badc8e4d254ee68b3668391059708a" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 42461 } } peers { permanent_uuid: "5c4f465260f94bdfac8e03636ec3d3f7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36671 } } peers { permanent_uuid: "8c3cfcc74d6e44e8a9e97df726938fbb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34227 } }
I20260806 01:14:16.370337 1842 leader_election.cc:288] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [CANDIDATE]: Term 1 election: Requested vote from peers 5c4f465260f94bdfac8e03636ec3d3f7 (127.25.236.130:36671), 8c3cfcc74d6e44e8a9e97df726938fbb (127.25.236.129:34227)
I20260806 01:14:16.370864 1532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94662fd4f14145ada538ebb6374d6737" candidate_uuid: "60badc8e4d254ee68b3668391059708a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4f465260f94bdfac8e03636ec3d3f7"
I20260806 01:14:16.370889 1340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94662fd4f14145ada538ebb6374d6737" candidate_uuid: "60badc8e4d254ee68b3668391059708a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3cfcc74d6e44e8a9e97df726938fbb"
I20260806 01:14:16.371125 1340 raft_consensus.cc:3037] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:16.371102 1532 raft_consensus.cc:3037] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:16.373618 1532 raft_consensus.cc:2454] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60badc8e4d254ee68b3668391059708a in term 1.
I20260806 01:14:16.373768 1340 raft_consensus.cc:2454] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60badc8e4d254ee68b3668391059708a in term 1.
I20260806 01:14:16.374060 1630 leader_election.cc:302] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c4f465260f94bdfac8e03636ec3d3f7, 60badc8e4d254ee68b3668391059708a; no voters:
I20260806 01:14:16.374472 1842 raft_consensus.cc:2781] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:16.374954 1842 raft_consensus.cc:686] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [term 1 LEADER]: Becoming Leader. State: Replica: 60badc8e4d254ee68b3668391059708a, State: Running, Role: LEADER
I20260806 01:14:16.375416 1842 consensus_queue.cc:237] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60badc8e4d254ee68b3668391059708a" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 42461 } } peers { permanent_uuid: "5c4f465260f94bdfac8e03636ec3d3f7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36671 } } peers { permanent_uuid: "8c3cfcc74d6e44e8a9e97df726938fbb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34227 } }
I20260806 01:14:16.380409 1110 catalog_manager.cc:5495] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a reported cstate change: term changed from 0 to 1, leader changed from <none> to 60badc8e4d254ee68b3668391059708a (127.25.236.131). New cstate: current_term: 1 leader_uuid: "60badc8e4d254ee68b3668391059708a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60badc8e4d254ee68b3668391059708a" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 42461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c4f465260f94bdfac8e03636ec3d3f7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3cfcc74d6e44e8a9e97df726938fbb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34227 } health_report { overall_health: UNKNOWN } } }
W20260806 01:14:16.393709 1607 tablet.cc:2369] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:14:16.398805 1802 tablet.cc:2369] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:14:16.414880 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.412881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59636 (local address 127.25.236.131:42461)
0806 01:14:16.413049 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:16.413054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:16.413072 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:14:16.413313 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:16.413317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:16.413402 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:16.413497 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:16.414411 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:16.414415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:16.414418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:16.414432 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:16.414464 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:16.414476 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:16.414587 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:16.414648 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 01:14:16.414694 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:14:16.424301 1532 raft_consensus.cc:1261] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7 [term 1 FOLLOWER]: Refusing update from remote peer 60badc8e4d254ee68b3668391059708a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:16.424301 1340 raft_consensus.cc:1261] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb [term 1 FOLLOWER]: Refusing update from remote peer 60badc8e4d254ee68b3668391059708a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:16.425917 1846 consensus_queue.cc:1035] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3cfcc74d6e44e8a9e97df726938fbb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:16.426512 1842 consensus_queue.cc:1035] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c4f465260f94bdfac8e03636ec3d3f7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:16.458546 1850 mvcc.cc:205] Tried to move back new op lower bound from 7315369395900698624 to 7315369395717791744. Current Snapshot: MvccSnapshot[applied={T|T < 7315369395900698624 or (T in {7315369395900698624})}]
I20260806 01:14:16.469034 1111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:16.469344 1111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:16.471750 1852 mvcc.cc:205] Tried to move back new op lower bound from 7315369395900698624 to 7315369395717791744. Current Snapshot: MvccSnapshot[applied={T|T < 7315369395900698624}]
I20260806 01:14:16.476440 1111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ea5aceb9800434d9ecd0d5c2d581e9a: Sending DeleteTablet for 3 replicas of tablet 94662fd4f14145ada538ebb6374d6737
I20260806 01:14:16.482954 1699 tablet_service.cc:1514] Processing DeleteTablet for tablet 94662fd4f14145ada538ebb6374d6737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:16 UTC) from {username='slave'} at 127.0.0.1:59626
I20260806 01:14:16.485000 1506 tablet_service.cc:1514] Processing DeleteTablet for tablet 94662fd4f14145ada538ebb6374d6737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:16 UTC) from {username='slave'} at 127.0.0.1:59046
I20260806 01:14:16.486629 1301 tablet_service.cc:1514] Processing DeleteTablet for tablet 94662fd4f14145ada538ebb6374d6737 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:16 UTC) from {username='slave'} at 127.0.0.1:48260
I20260806 01:14:16.491122 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 1201
I20260806 01:14:16.499034 1865 tablet_replica.cc:330] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a: stopping tablet replica
I20260806 01:14:16.499740 1865 raft_consensus.cc:2229] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:14:16.500464 1865 raft_consensus.cc:2258] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:16.501544 1829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.493427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48274 (local address 127.25.236.129:34227)
0806 01:14:16.493720 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:16.493730 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:16.498005 (+ 4275us) server_negotiation.cc:407] Connection header received
0806 01:14:16.498354 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:16.498363 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:16.498496 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:16.498592 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:16.500430 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:16.500439 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:16.500447 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:16.500475 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:16.500541 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:16.500565 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:16.500695 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:16.500991 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 01:14:16.501071 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260806 01:14:16.499794 1867 tablet_replica.cc:330] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb: stopping tablet replica
I20260806 01:14:16.502811 1395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:16.503895 1865 ts_tablet_manager.cc:1905] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:16.503921 1866 tablet_replica.cc:330] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7: stopping tablet replica
I20260806 01:14:16.504415 1866 raft_consensus.cc:2229] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:16.505074 1866 raft_consensus.cc:2258] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:16.507404 1866 ts_tablet_manager.cc:1905] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:16.512279 1865 ts_tablet_manager.cc:1918] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:16.512475 1865 log.cc:1196] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/94662fd4f14145ada538ebb6374d6737
I20260806 01:14:16.513053 1865 ts_tablet_manager.cc:1939] T 94662fd4f14145ada538ebb6374d6737 P 60badc8e4d254ee68b3668391059708a: Deleting consensus metadata
I20260806 01:14:16.515064 1093 catalog_manager.cc:4841] TS 60badc8e4d254ee68b3668391059708a (127.25.236.131:42461): tablet 94662fd4f14145ada538ebb6374d6737 (table test-table [id=9bb8ae2e7fb046d092eb3d106c7824ec]) successfully deleted
I20260806 01:14:16.518692 1866 ts_tablet_manager.cc:1918] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:16.519047 1866 log.cc:1196] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/94662fd4f14145ada538ebb6374d6737
I20260806 01:14:16.519901 1866 ts_tablet_manager.cc:1939] T 94662fd4f14145ada538ebb6374d6737 P 5c4f465260f94bdfac8e03636ec3d3f7: Deleting consensus metadata
I20260806 01:14:16.521783 1089 catalog_manager.cc:4841] TS 5c4f465260f94bdfac8e03636ec3d3f7 (127.25.236.130:36671): tablet 94662fd4f14145ada538ebb6374d6737 (table test-table [id=9bb8ae2e7fb046d092eb3d106c7824ec]) successfully deleted
I20260806 01:14:16.502964 1867 raft_consensus.cc:2229] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:16.630854 1867 raft_consensus.cc:2258] T 94662fd4f14145ada538ebb6374d6737 P 8c3cfcc74d6e44e8a9e97df726938fbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:16.631067 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 1201
W20260806 01:14:16.659101 1089 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.236.129:34227 (error 108)
I20260806 01:14:16.660271 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 1409
I20260806 01:14:16.665856 1826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.662452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59060 (local address 127.25.236.130:36671)
0806 01:14:16.662682 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:16.662691 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:16.662716 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:14:16.663064 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:16.663074 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:16.663193 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:16.663307 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:16.664787 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:16.664794 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:16.664799 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:16.664820 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:16.664861 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:16.664880 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:16.665009 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:16.665463 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 01:14:16.665529 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:14:16.666651 1591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:16.666859 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.665750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34227 (local address 127.0.0.1:48284)
0806 01:14:16.666033 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:16.666694 (+ 661us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
W20260806 01:14:16.667204 1089 catalog_manager.cc:4584] TS 8c3cfcc74d6e44e8a9e97df726938fbb (127.25.236.129:34227): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94662fd4f14145ada538ebb6374d6737: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34227: connect: Connection refused (error 111)
I20260806 01:14:16.721519 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.720828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34227 (local address 127.0.0.1:48292)
0806 01:14:16.721196 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:16.721331 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216}
I20260806 01:14:16.723259 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.722691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34227 (local address 127.0.0.1:48308)
0806 01:14:16.722886 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:16.722983 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 01:14:16.769436 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 1409
I20260806 01:14:16.784164 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.783580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34227 (local address 127.0.0.1:48324)
0806 01:14:16.783839 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:16.783951 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 01:14:16.785818 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.784985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34227 (local address 127.0.0.1:48326)
0806 01:14:16.785420 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:16.785590 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 01:14:16.796677 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 1611
I20260806 01:14:16.800818 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.797676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59646 (local address 127.25.236.131:42461)
0806 01:14:16.797857 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:16.797861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:16.797937 (+ 76us) server_negotiation.cc:407] Connection header received
0806 01:14:16.798522 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:16.798527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:16.798583 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:16.798679 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:16.799818 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:16.799825 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:16.799829 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:16.799854 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:16.799906 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:16.799923 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:16.800064 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:16.800417 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 01:14:16.800515 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:14:16.801942 1793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:16.866957 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.866420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34227 (local address 127.0.0.1:48340)
0806 01:14:16.866670 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:16.866776 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 01:14:16.868253 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:16.867853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34227 (local address 127.0.0.1:48342)
0806 01:14:16.868060 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:16.868139 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 01:14:16.931409 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 1611
I20260806 01:14:16.966473 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 1065
I20260806 01:14:16.967727 1160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:17.084604 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:17.083808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34227 (local address 127.0.0.1:48352)
0806 01:14:17.084044 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:17.084167 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 01:14:17.084802 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 1065
2026-08-06T01:14:17Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3184 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 01:14:17.136536 26546 mini_oidc.cc:152] Starting JWKS server
I20260806 01:14:17.140591 26546 webserver.cc:460] Webserver started at https://127.0.0.1:33873/ using document root <none> and password file <none>
I20260806 01:14:17.141006 26546 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 01:14:17.141822 26546 webserver.cc:460] Webserver started at http://127.0.0.1:46705/ using document root <none> and password file <none>
2026-08-06T01:14:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:14:17Z Disabled control of system clock
I20260806 01:14:17.181087 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:37837
--webserver_interface=127.25.236.190
--webserver_port=0
--builtin_ntp_servers=127.25.236.148:37423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:37837
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33873/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/testchainca.cert with env {}
W20260806 01:14:17.567546 1886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:17.567847 1886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:17.568010 1886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:17.576033 1886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:17.576179 1886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:17.576224 1886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:17.576380 1886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:17.576490 1886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 01:14:17.576639 1886 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 01:14:17.576726 1886 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33873/jwks.json
I20260806 01:14:17.588413 1886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:37423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:37837
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:37837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33873/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:17.589902 1886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:17.591315 1886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:17.605693 1894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:17.605634 1897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:17.605747 1895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:17.612414 1886 server_base.cc:1034] running on GCE node
I20260806 01:14:17.613461 1886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:17.615573 1886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:17.616981 1886 hybrid_clock.cc:648] HybridClock initialized: now 1785978857616893 us; error 65 us; skew 500 ppm
I20260806 01:14:17.617547 1886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:14:17.627779 1886 webserver.cc:460] Webserver started at http://127.25.236.190:41183/ using document root <none> and password file <none>
I20260806 01:14:17.628722 1886 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:17.628873 1886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:17.629235 1886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:17.632717 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "89dc3ee32baa470db936407ae12f1c4c"
format_stamp: "Formatted at 2026-08-06 01:14:17 on dist-test-slave-7nm7"
server_key: "b47e649b025d2ba503108b6ace6d8f64"
server_key_iv: "4270bd0b3cd262af771f976d8a3b6d52"
server_key_version: "encryptionkey@0"
I20260806 01:14:17.633697 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "89dc3ee32baa470db936407ae12f1c4c"
format_stamp: "Formatted at 2026-08-06 01:14:17 on dist-test-slave-7nm7"
server_key: "b47e649b025d2ba503108b6ace6d8f64"
server_key_iv: "4270bd0b3cd262af771f976d8a3b6d52"
server_key_version: "encryptionkey@0"
I20260806 01:14:17.639307 1886 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:14:17.643466 1907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:17.645233 1886 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:14:17.645470 1886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "89dc3ee32baa470db936407ae12f1c4c"
format_stamp: "Formatted at 2026-08-06 01:14:17 on dist-test-slave-7nm7"
server_key: "b47e649b025d2ba503108b6ace6d8f64"
server_key_iv: "4270bd0b3cd262af771f976d8a3b6d52"
server_key_version: "encryptionkey@0"
I20260806 01:14:17.645718 1886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:17.673563 1886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:17.674585 1886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:17.674928 1886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:17.698530 1983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:37837 every 8 connection(s)
I20260806 01:14:17.698526 1886 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:37837
I20260806 01:14:17.700075 1886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:14:17.704646 1985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:17.710148 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 1886
I20260806 01:14:17.710428 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:14:17.711088 26546 external_mini_cluster.cc:1428] Setting key 9e544eb12877018f293aa140e447a54e
I20260806 01:14:17.717995 1985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c: Bootstrap starting.
I20260806 01:14:17.722429 1985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:17.723503 1985 log.cc:824] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:17.726967 1985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c: No bootstrap required, opened a new log
I20260806 01:14:17.730762 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:17.713863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49438 (local address 127.25.236.190:37837)
0806 01:14:17.715546 (+ 1683us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:17.715562 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:17.715615 (+ 53us) server_negotiation.cc:407] Connection header received
0806 01:14:17.718161 (+ 2546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:17.718195 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:17.718929 (+ 734us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:17.719402 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:17.720033 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:17.722015 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:17.724465 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:17.725457 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:17.725739 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:17.727595 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:17.727603 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:17.727618 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:17.728420 (+ 802us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:17.728573 (+ 153us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:17.728616 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:17.728846 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:17.729238 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 01:14:17.729615 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":963,"thread_start_us":574,"threads_started":1}
I20260806 01:14:17.735466 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:14:17.736866 1985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dc3ee32baa470db936407ae12f1c4c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37837 } }
I20260806 01:14:17.737349 1985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:17.737470 1985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89dc3ee32baa470db936407ae12f1c4c, State: Initialized, Role: FOLLOWER
I20260806 01:14:17.738188 1985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dc3ee32baa470db936407ae12f1c4c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37837 } }
I20260806 01:14:17.738535 1985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:14:17.738700 1985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:14:17.738920 1985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:17.741462 1985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dc3ee32baa470db936407ae12f1c4c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37837 } }
I20260806 01:14:17.742009 1985 leader_election.cc:302] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89dc3ee32baa470db936407ae12f1c4c; no voters:
I20260806 01:14:17.742605 1985 leader_election.cc:288] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:14:17.742805 1993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:17.743537 1993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [term 1 LEADER]: Becoming Leader. State: Replica: 89dc3ee32baa470db936407ae12f1c4c, State: Running, Role: LEADER
I20260806 01:14:17.744206 1993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dc3ee32baa470db936407ae12f1c4c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37837 } }
I20260806 01:14:17.744997 1985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:17.747478 1994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89dc3ee32baa470db936407ae12f1c4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dc3ee32baa470db936407ae12f1c4c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37837 } } }
I20260806 01:14:17.747555 1996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89dc3ee32baa470db936407ae12f1c4c. Latest consensus state: current_term: 1 leader_uuid: "89dc3ee32baa470db936407ae12f1c4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dc3ee32baa470db936407ae12f1c4c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37837 } } }
I20260806 01:14:17.747892 1994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:17.747956 1996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:17.748847 2002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:17.756217 2002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:17.767426 2002 catalog_manager.cc:1347] Generated new cluster ID: 256e875fbf4f41c98bb8f34356f10dfc
I20260806 01:14:17.767617 2002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:17.781955 2002 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:14:17.783419 2002 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:17.790654 2002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89dc3ee32baa470db936407ae12f1c4c: Generated new TSK 0
I20260806 01:14:17.791550 2002 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 01:14:17.845458 2027 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 01:14:17.845821 2027 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 01:14:17.848767 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:17.819595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49444 (local address 127.25.236.190:37837)
0806 01:14:17.819835 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:17.819842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:17.820609 (+ 767us) server_negotiation.cc:407] Connection header received
0806 01:14:17.820960 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:17.820967 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:17.821075 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:17.821247 (+ 172us) server_negotiation.cc:226] Negotiated authn=JWT
0806 01:14:17.822171 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:17.823460 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:17.825058 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:17.826057 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:17.826276 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:17.827393 (+ 1117us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 01:14:17.847637 (+ 20244us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 01:14:17.848436 (+ 799us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49444: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 01:14:17.848920 1990 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49444: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 01:14:17.850859 2026 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:17.819735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:37837 (local address 127.0.0.1:49444)
0806 01:14:17.820486 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:17.820514 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:17.820673 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:17.821420 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:17.821426 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:17.821443 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0806 01:14:17.821921 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:17.821932 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:17.823664 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:17.823669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:17.824896 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:17.824910 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:17.826504 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:17.826512 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:17.826707 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:17.826816 (+ 109us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 01:14:17.848431 (+ 21615us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 01:14:17.848789 (+ 358us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.236.190:37837: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":425,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.25.236.190:37837: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 01:14:17.852669 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 1886
I20260806 01:14:17.853936 1976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:17.955075 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 1886
2026-08-06T01:14:17Z chronyd exiting
I20260806 01:14:18.017755 26546 test_util.cc:175] -----------------------------------------------
I20260806 01:14:18.017895 26546 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978805316623-26546-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (887 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 01:14:18.022814 26546 mini_oidc.cc:152] Starting JWKS server
I20260806 01:14:18.024904 26546 webserver.cc:460] Webserver started at https://127.0.0.1:40059/ using document root <none> and password file <none>
I20260806 01:14:18.025177 26546 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 01:14:18.025878 26546 webserver.cc:460] Webserver started at http://127.0.0.1:38153/ using document root <none> and password file <none>
2026-08-06T01:14:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:14:18Z Disabled control of system clock
I20260806 01:14:18.066592 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:39407
--webserver_interface=127.25.236.190
--webserver_port=0
--builtin_ntp_servers=127.25.236.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:39407
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40059/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/testchainca.cert with env {}
W20260806 01:14:18.454866 2039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:18.455189 2039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:18.455281 2039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:18.462806 2039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:18.462934 2039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:18.462975 2039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:18.463143 2039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:18.463256 2039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 01:14:18.463336 2039 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 01:14:18.463385 2039 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40059/jwks.json
I20260806 01:14:18.476869 2039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:39407
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:39407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40059/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:18.478259 2039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:18.479552 2039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:18.491845 2049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:18.491799 2050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:18.491882 2052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:18.493211 2039 server_base.cc:1034] running on GCE node
I20260806 01:14:18.494066 2039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:18.495654 2039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:18.496924 2039 hybrid_clock.cc:648] HybridClock initialized: now 1785978858496876 us; error 55 us; skew 500 ppm
I20260806 01:14:18.497370 2039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:14:18.500464 2039 webserver.cc:460] Webserver started at http://127.25.236.190:44433/ using document root <none> and password file <none>
I20260806 01:14:18.501225 2039 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:18.501374 2039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:18.501737 2039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:18.505151 2039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "f874360c8b9348ee8c0ccbcc52d7485a"
format_stamp: "Formatted at 2026-08-06 01:14:18 on dist-test-slave-7nm7"
server_key: "46e0f0cf0814879d4bb8a9ea44abef83"
server_key_iv: "681e914f8af2425cb2040379aca545a5"
server_key_version: "encryptionkey@0"
I20260806 01:14:18.506045 2039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "f874360c8b9348ee8c0ccbcc52d7485a"
format_stamp: "Formatted at 2026-08-06 01:14:18 on dist-test-slave-7nm7"
server_key: "46e0f0cf0814879d4bb8a9ea44abef83"
server_key_iv: "681e914f8af2425cb2040379aca545a5"
server_key_version: "encryptionkey@0"
I20260806 01:14:18.512274 2039 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 01:14:18.516808 2062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:18.518514 2039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:14:18.518719 2039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "f874360c8b9348ee8c0ccbcc52d7485a"
format_stamp: "Formatted at 2026-08-06 01:14:18 on dist-test-slave-7nm7"
server_key: "46e0f0cf0814879d4bb8a9ea44abef83"
server_key_iv: "681e914f8af2425cb2040379aca545a5"
server_key_version: "encryptionkey@0"
I20260806 01:14:18.518972 2039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:18.534806 2039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:18.535742 2039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:18.536113 2039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:18.558005 2039 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:39407
I20260806 01:14:18.558043 2139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:39407 every 8 connection(s)
I20260806 01:14:18.559428 2039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:14:18.563136 2145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:18.564512 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 2039
I20260806 01:14:18.564721 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:14:18.565600 26546 external_mini_cluster.cc:1428] Setting key 6ccadae5223eadb7619283c06e81c5a9
I20260806 01:14:18.577590 2145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a: Bootstrap starting.
I20260806 01:14:18.579643 2148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:18.567650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45416 (local address 127.25.236.190:39407)
0806 01:14:18.568717 (+ 1067us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:18.568731 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:18.568774 (+ 43us) server_negotiation.cc:407] Connection header received
0806 01:14:18.570393 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:18.570415 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:18.570874 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:18.571190 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:18.571723 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:18.572962 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:18.574851 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:18.575650 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:18.575809 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:18.577098 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:18.577106 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:18.577130 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:18.577780 (+ 650us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:18.577879 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:18.577903 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:18.578012 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:18.578381 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 01:14:18.578690 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":424,"threads_started":1}
I20260806 01:14:18.581108 2145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:18.582432 2145 log.cc:824] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:18.583801 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:14:18.585632 2145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a: No bootstrap required, opened a new log
I20260806 01:14:18.594807 2145 raft_consensus.cc:348] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f874360c8b9348ee8c0ccbcc52d7485a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39407 } }
I20260806 01:14:18.595211 2145 raft_consensus.cc:374] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:18.595348 2145 raft_consensus.cc:729] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f874360c8b9348ee8c0ccbcc52d7485a, State: Initialized, Role: FOLLOWER
I20260806 01:14:18.596014 2145 consensus_queue.cc:260] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f874360c8b9348ee8c0ccbcc52d7485a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39407 } }
I20260806 01:14:18.596454 2145 raft_consensus.cc:388] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:14:18.596653 2145 raft_consensus.cc:482] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:14:18.596870 2145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:18.599996 2145 raft_consensus.cc:504] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f874360c8b9348ee8c0ccbcc52d7485a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39407 } }
I20260806 01:14:18.600764 2145 leader_election.cc:302] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f874360c8b9348ee8c0ccbcc52d7485a; no voters:
I20260806 01:14:18.601495 2145 leader_election.cc:288] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:14:18.601737 2150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:18.603060 2150 raft_consensus.cc:686] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [term 1 LEADER]: Becoming Leader. State: Replica: f874360c8b9348ee8c0ccbcc52d7485a, State: Running, Role: LEADER
I20260806 01:14:18.603762 2150 consensus_queue.cc:237] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f874360c8b9348ee8c0ccbcc52d7485a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39407 } }
I20260806 01:14:18.604209 2145 sys_catalog.cc:564] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:18.607308 2152 sys_catalog.cc:455] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f874360c8b9348ee8c0ccbcc52d7485a. Latest consensus state: current_term: 1 leader_uuid: "f874360c8b9348ee8c0ccbcc52d7485a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f874360c8b9348ee8c0ccbcc52d7485a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39407 } } }
I20260806 01:14:18.607694 2151 sys_catalog.cc:455] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f874360c8b9348ee8c0ccbcc52d7485a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f874360c8b9348ee8c0ccbcc52d7485a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39407 } } }
I20260806 01:14:18.608013 2152 sys_catalog.cc:458] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:18.608022 2151 sys_catalog.cc:458] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:18.610577 2160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:18.616288 2160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:18.626201 2160 catalog_manager.cc:1347] Generated new cluster ID: 20468f50cef64a93b96dbb5b0ab182f1
I20260806 01:14:18.626374 2160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:18.643579 2160 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:14:18.645581 2160 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:18.658380 2160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f874360c8b9348ee8c0ccbcc52d7485a: Generated new TSK 0
I20260806 01:14:18.659359 2160 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 01:14:18.704416 2183 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 01:14:18.704560 2183 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 01:14:18.705916 2148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:18.683744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45424 (local address 127.25.236.190:39407)
0806 01:14:18.683949 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:18.683957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:18.684347 (+ 390us) server_negotiation.cc:407] Connection header received
0806 01:14:18.684582 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:18.684588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:18.684694 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:18.684891 (+ 197us) server_negotiation.cc:226] Negotiated authn=JWT
0806 01:14:18.685564 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:18.686782 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:18.688238 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:18.688962 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:18.689099 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:18.690052 (+ 953us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 01:14:18.705122 (+ 15070us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40059/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 01:14:18.705561 (+ 439us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45424: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40059/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":81}
W20260806 01:14:18.706058 2148 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45424: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40059/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 01:14:18.706102 2181 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:18.683558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39407 (local address 127.0.0.1:45424)
0806 01:14:18.684228 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:18.684246 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:18.684377 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:18.684976 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:18.684984 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:18.685001 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0806 01:14:18.685370 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:18.685387 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:18.686968 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:18.686973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:18.688067 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:18.688079 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:18.689334 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:18.689343 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:18.689572 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:18.689631 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 01:14:18.705456 (+ 15825us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40059/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 01:14:18.705757 (+ 301us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.236.190:39407: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40059/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":357,"threads_started":1}
I20260806 01:14:18.709052 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 2039
I20260806 01:14:18.710561 2135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:18.811566 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 2039
2026-08-06T01:14:18Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (859 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 01:14:18.881397 26546 mini_oidc.cc:152] Starting JWKS server
I20260806 01:14:18.883517 26546 webserver.cc:460] Webserver started at https://127.0.0.1:38207/ using document root <none> and password file <none>
I20260806 01:14:18.883805 26546 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 01:14:18.884634 26546 webserver.cc:460] Webserver started at http://127.0.0.1:35421/ using document root <none> and password file <none>
2026-08-06T01:14:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:14:18Z Disabled control of system clock
I20260806 01:14:18.931244 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:38073
--webserver_interface=127.25.236.190
--webserver_port=0
--builtin_ntp_servers=127.25.236.148:41775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:38073
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38207/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 01:14:19.312073 2194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:19.312424 2194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:19.312625 2194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:19.321117 2194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:19.321388 2194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:19.321496 2194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:19.321568 2194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:19.321664 2194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 01:14:19.321928 2194 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 01:14:19.322049 2194 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38207/jwks.json
I20260806 01:14:19.333361 2194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:38073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:38073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38207/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:19.334911 2194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:19.336767 2194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:19.350268 2200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:19.350184 2202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:19.350419 2205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:19.352088 2194 server_base.cc:1034] running on GCE node
I20260806 01:14:19.352936 2194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:19.354673 2194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:19.355993 2194 hybrid_clock.cc:648] HybridClock initialized: now 1785978859355927 us; error 51 us; skew 500 ppm
I20260806 01:14:19.356436 2194 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:14:19.359285 2194 webserver.cc:460] Webserver started at http://127.25.236.190:38281/ using document root <none> and password file <none>
I20260806 01:14:19.360050 2194 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:19.360195 2194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:19.360555 2194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:19.364140 2194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "cc5aa2b4ab7d4f3385af9d3aef534c0a"
format_stamp: "Formatted at 2026-08-06 01:14:19 on dist-test-slave-7nm7"
server_key: "9ed75b7efe353420adb62742c7532268"
server_key_iv: "181b427ac09c42c0d4cd310b1edf15d4"
server_key_version: "encryptionkey@0"
I20260806 01:14:19.365104 2194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "cc5aa2b4ab7d4f3385af9d3aef534c0a"
format_stamp: "Formatted at 2026-08-06 01:14:19 on dist-test-slave-7nm7"
server_key: "9ed75b7efe353420adb62742c7532268"
server_key_iv: "181b427ac09c42c0d4cd310b1edf15d4"
server_key_version: "encryptionkey@0"
I20260806 01:14:19.371025 2194 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 01:14:19.375030 2214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:19.377508 2194 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 01:14:19.377859 2194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "cc5aa2b4ab7d4f3385af9d3aef534c0a"
format_stamp: "Formatted at 2026-08-06 01:14:19 on dist-test-slave-7nm7"
server_key: "9ed75b7efe353420adb62742c7532268"
server_key_iv: "181b427ac09c42c0d4cd310b1edf15d4"
server_key_version: "encryptionkey@0"
I20260806 01:14:19.378188 2194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:19.401397 2194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:19.402383 2194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:19.402722 2194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:19.424918 2194 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:38073
I20260806 01:14:19.424942 2295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:38073 every 8 connection(s)
I20260806 01:14:19.426525 2194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:14:19.430795 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 2194
I20260806 01:14:19.431023 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:14:19.431838 2297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:19.431921 26546 external_mini_cluster.cc:1428] Setting key b4fd7154d41f1e0a879c0d68ed790842
I20260806 01:14:19.447420 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:19.433945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48286 (local address 127.25.236.190:38073)
0806 01:14:19.435042 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:19.435058 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:19.435103 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:14:19.437176 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:19.437200 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:19.437921 (+ 721us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:19.438493 (+ 572us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:19.438965 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:19.440545 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:19.442410 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:19.443175 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:19.443328 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:19.444864 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:19.444874 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:19.444893 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:19.445522 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:19.445607 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:19.445634 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:19.445738 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:19.446024 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 01:14:19.446426 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":449,"threads_started":1}
I20260806 01:14:19.448173 2297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a: Bootstrap starting.
I20260806 01:14:19.451617 2297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:19.452256 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:14:19.452852 2297 log.cc:824] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:19.456087 2297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a: No bootstrap required, opened a new log
I20260806 01:14:19.464814 2297 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5aa2b4ab7d4f3385af9d3aef534c0a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38073 } }
I20260806 01:14:19.465379 2297 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:19.465669 2297 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc5aa2b4ab7d4f3385af9d3aef534c0a, State: Initialized, Role: FOLLOWER
I20260806 01:14:19.466900 2297 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5aa2b4ab7d4f3385af9d3aef534c0a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38073 } }
I20260806 01:14:19.467293 2297 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:14:19.467485 2297 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:14:19.467711 2297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:19.470625 2297 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5aa2b4ab7d4f3385af9d3aef534c0a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38073 } }
I20260806 01:14:19.471649 2297 leader_election.cc:302] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc5aa2b4ab7d4f3385af9d3aef534c0a; no voters:
I20260806 01:14:19.472576 2297 leader_election.cc:288] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:14:19.472810 2302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:19.474690 2302 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [term 1 LEADER]: Becoming Leader. State: Replica: cc5aa2b4ab7d4f3385af9d3aef534c0a, State: Running, Role: LEADER
I20260806 01:14:19.475332 2302 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5aa2b4ab7d4f3385af9d3aef534c0a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38073 } }
I20260806 01:14:19.475852 2297 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:19.478554 2306 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc5aa2b4ab7d4f3385af9d3aef534c0a. Latest consensus state: current_term: 1 leader_uuid: "cc5aa2b4ab7d4f3385af9d3aef534c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5aa2b4ab7d4f3385af9d3aef534c0a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38073 } } }
I20260806 01:14:19.479055 2306 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:19.479529 2303 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc5aa2b4ab7d4f3385af9d3aef534c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5aa2b4ab7d4f3385af9d3aef534c0a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38073 } } }
I20260806 01:14:19.479925 2303 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:19.484899 2315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:19.491408 2315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:19.503182 2315 catalog_manager.cc:1347] Generated new cluster ID: c822da6eb7994a4c85eb3fbac0c0ff9c
I20260806 01:14:19.503332 2315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:19.516909 2315 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:14:19.518683 2315 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:19.527998 2315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc5aa2b4ab7d4f3385af9d3aef534c0a: Generated new TSK 0
I20260806 01:14:19.529299 2315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:14:19.567518 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:19.557961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48294 (local address 127.25.236.190:38073)
0806 01:14:19.558165 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:19.558172 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:19.558537 (+ 365us) server_negotiation.cc:407] Connection header received
0806 01:14:19.558774 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:19.558780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:19.558878 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:19.559053 (+ 175us) server_negotiation.cc:226] Negotiated authn=JWT
0806 01:14:19.559631 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:19.560838 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:19.562362 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:19.563069 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:19.563230 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:19.563900 (+ 670us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 01:14:19.566730 (+ 2830us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38207/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 01:14:19.567137 (+ 407us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48294: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38207/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":76}
W20260806 01:14:19.567674 2300 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48294: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38207/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 01:14:19.569177 2335 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:19.557779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:38073 (local address 127.0.0.1:48294)
0806 01:14:19.558433 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:19.558451 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:19.558576 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:19.559110 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:19.559115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:19.559125 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0806 01:14:19.559462 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:19.559470 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:19.560999 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:19.561004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:19.562159 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:19.562169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:19.563398 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:19.563408 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:19.563541 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:19.563580 (+ 39us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 01:14:19.566970 (+ 3390us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38207/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 01:14:19.568754 (+ 1784us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.236.190:38073: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38207/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":1440,"thread_start_us":333,"threads_started":1}
I20260806 01:14:19.570402 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 2194
I20260806 01:14:19.572362 2290 generic_service.cc:196] Checking for leaks (request via RPC)
E20260806 01:14:19.580130 2338 webserver.cc:533] Webserver: sslize failed:
E20260806 01:14:19.580236 2338 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 01:14:19.671523 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 2194
2026-08-06T01:14:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (854 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T01:14:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:14:19Z Disabled control of system clock
I20260806 01:14:19.774735 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:37683
--webserver_interface=127.25.236.190
--webserver_port=0
--builtin_ntp_servers=127.25.236.148:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:37683
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/testchainca.cert with env {}
W20260806 01:14:20.161494 2343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:20.161814 2343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:20.161906 2343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:20.169286 2343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:20.169412 2343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:20.169451 2343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:20.169505 2343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:20.169560 2343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 01:14:20.169620 2343 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 01:14:20.169660 2343 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 01:14:20.181097 2343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:37683
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:37683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:20.182533 2343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:20.183897 2343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:20.196754 2355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:20.196966 2353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:20.197136 2352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:20.197834 2343 server_base.cc:1034] running on GCE node
I20260806 01:14:20.198707 2343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:20.200088 2343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:20.201431 2343 hybrid_clock.cc:648] HybridClock initialized: now 1785978860201349 us; error 50 us; skew 500 ppm
I20260806 01:14:20.202081 2343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:14:20.204643 2343 webserver.cc:460] Webserver started at http://127.25.236.190:46825/ using document root <none> and password file <none>
I20260806 01:14:20.205394 2343 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:20.205542 2343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:20.205875 2343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:20.209631 2343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "e3edd7ef865544d0b31998ede5b36073"
format_stamp: "Formatted at 2026-08-06 01:14:20 on dist-test-slave-7nm7"
server_key: "cc9b361bfcd2e6087db8e35b4e244b7b"
server_key_iv: "560449202c4e6104603f8d59668521ef"
server_key_version: "encryptionkey@0"
I20260806 01:14:20.210649 2343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "e3edd7ef865544d0b31998ede5b36073"
format_stamp: "Formatted at 2026-08-06 01:14:20 on dist-test-slave-7nm7"
server_key: "cc9b361bfcd2e6087db8e35b4e244b7b"
server_key_iv: "560449202c4e6104603f8d59668521ef"
server_key_version: "encryptionkey@0"
I20260806 01:14:20.216688 2343 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 01:14:20.220770 2366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:20.222532 2343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:14:20.222741 2343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "e3edd7ef865544d0b31998ede5b36073"
format_stamp: "Formatted at 2026-08-06 01:14:20 on dist-test-slave-7nm7"
server_key: "cc9b361bfcd2e6087db8e35b4e244b7b"
server_key_iv: "560449202c4e6104603f8d59668521ef"
server_key_version: "encryptionkey@0"
I20260806 01:14:20.222996 2343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:20.251585 2343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:20.252692 2343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:20.253335 2343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:20.279510 2438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:37683 every 8 connection(s)
I20260806 01:14:20.279593 2343 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:37683
I20260806 01:14:20.281524 2343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:14:20.283598 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 2343
I20260806 01:14:20.283838 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:14:20.284778 26546 external_mini_cluster.cc:1428] Setting key e6b11c31d6f8cc225792c971640e6151
I20260806 01:14:20.287808 2439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:20.301816 2446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:20.286953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58878 (local address 127.25.236.190:37683)
0806 01:14:20.288127 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:20.288139 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:20.288188 (+ 49us) server_negotiation.cc:407] Connection header received
0806 01:14:20.289921 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:20.289942 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:20.290542 (+ 600us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:20.290914 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:20.291568 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:20.293007 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:20.295025 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:20.295882 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:20.296026 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:20.298171 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:20.298192 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:20.298211 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:14:20.299056 (+ 845us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:20.299387 (+ 331us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:14:20.299420 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:20.299539 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:20.300066 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 01:14:20.300699 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":447,"threads_started":1}
I20260806 01:14:20.302520 2439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073: Bootstrap starting.
I20260806 01:14:20.306680 2439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:20.307294 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:14:20.307868 2439 log.cc:824] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:20.312444 2439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073: No bootstrap required, opened a new log
I20260806 01:14:20.322898 2439 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3edd7ef865544d0b31998ede5b36073" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37683 } }
I20260806 01:14:20.323359 2439 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:20.323472 2439 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3edd7ef865544d0b31998ede5b36073, State: Initialized, Role: FOLLOWER
I20260806 01:14:20.324115 2439 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3edd7ef865544d0b31998ede5b36073" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37683 } }
I20260806 01:14:20.324335 2439 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:14:20.324517 2439 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:14:20.324714 2439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:20.327400 2439 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3edd7ef865544d0b31998ede5b36073" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37683 } }
I20260806 01:14:20.328039 2439 leader_election.cc:302] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3edd7ef865544d0b31998ede5b36073; no voters:
I20260806 01:14:20.328704 2439 leader_election.cc:288] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:14:20.328894 2450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:20.329641 2450 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [term 1 LEADER]: Becoming Leader. State: Replica: e3edd7ef865544d0b31998ede5b36073, State: Running, Role: LEADER
I20260806 01:14:20.330334 2450 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3edd7ef865544d0b31998ede5b36073" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37683 } }
I20260806 01:14:20.331076 2439 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:20.333949 2451 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3edd7ef865544d0b31998ede5b36073" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3edd7ef865544d0b31998ede5b36073" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37683 } } }
I20260806 01:14:20.334059 2452 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3edd7ef865544d0b31998ede5b36073. Latest consensus state: current_term: 1 leader_uuid: "e3edd7ef865544d0b31998ede5b36073" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3edd7ef865544d0b31998ede5b36073" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37683 } } }
I20260806 01:14:20.334618 2451 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:20.334705 2452 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:20.341190 2457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:20.347990 2457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:20.360845 2457 catalog_manager.cc:1347] Generated new cluster ID: 8e93e3acfd1c4f42a527f66dea3f151e
I20260806 01:14:20.360996 2457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:20.377059 2457 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:14:20.379696 2457 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:20.388371 2457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3edd7ef865544d0b31998ede5b36073: Generated new TSK 0
I20260806 01:14:20.389300 2457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:14:20.508674 2446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:20.410320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58892 (local address 127.25.236.190:37683)
0806 01:14:20.410576 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:20.410586 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:20.411171 (+ 585us) server_negotiation.cc:407] Connection header received
0806 01:14:20.411637 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:20.411644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:20.411764 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:20.411999 (+ 235us) server_negotiation.cc:226] Negotiated authn=JWT
0806 01:14:20.412723 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:20.413529 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:20.415712 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:20.416169 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:20.416301 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:20.417002 (+ 701us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 01:14:20.507789 (+ 90787us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 01:14:20.508283 (+ 494us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58892: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":75}
W20260806 01:14:20.508706 2488 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:20.410203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:37683 (local address 127.0.0.1:58892)
0806 01:14:20.411005 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:20.411029 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:20.411212 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:20.412066 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:20.412072 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:20.412087 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0806 01:14:20.412466 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:20.412493 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:20.413698 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:20.413704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:20.415528 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:20.415546 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:20.416391 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:20.416399 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:20.416614 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:20.416668 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 01:14:20.508111 (+ 91443us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 01:14:20.508376 (+ 265us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.236.190:37683: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":364,"threads_started":1}
W20260806 01:14:20.508838 2446 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58892: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260806 01:14:20.510037 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 2343
I20260806 01:14:20.511636 2432 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:20.618912 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 2343
2026-08-06T01:14:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (928 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (465 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (463 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:14:21 dist-test-slave-7nm7 krb5kdc[2513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:14:21 dist-test-slave-7nm7 krb5kdc[2513](info): set up 2 sockets
Aug 06 01:14:21 dist-test-slave-7nm7 krb5kdc[2513](info): commencing operation
krb5kdc: starting...
W20260806 01:14:22.948734 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.354s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:14:23 dist-test-slave-7nm7 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:14:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:14:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:14:23.237996 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:33351
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:38191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:33351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:23.607388 2528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:23.607713 2528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:23.607868 2528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:23.615442 2528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:23.615566 2528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:23.615615 2528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:23.615674 2528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:23.615720 2528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:14:23.625778 2528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:38191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:33351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:33351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:23.627187 2528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:23.628544 2528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:23.640849 2540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:23.640851 2534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:23.640872 2536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:23.641911 2528 server_base.cc:1034] running on GCE node
I20260806 01:14:23.642838 2528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:23.644345 2528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:23.645617 2528 hybrid_clock.cc:648] HybridClock initialized: now 1785978863645570 us; error 43 us; skew 500 ppm
I20260806 01:14:23.646106 2528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:23 dist-test-slave-7nm7 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978863, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:23.652905 2528 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:14:23.654568 2528 webserver.cc:460] Webserver started at http://127.25.236.190:41663/ using document root <none> and password file <none>
I20260806 01:14:23.655256 2528 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:23.655360 2528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:23.655618 2528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:23.658739 2528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "af5eca9b3ee1465084d78325c7932858"
format_stamp: "Formatted at 2026-08-06 01:14:23 on dist-test-slave-7nm7"
server_key: "6fb217cf20d2f0d65ffa479ae4d39709"
server_key_iv: "7e006674bda7dc7d4ea4304c3387d28b"
server_key_version: "encryptionkey@0"
I20260806 01:14:23.659487 2528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "af5eca9b3ee1465084d78325c7932858"
format_stamp: "Formatted at 2026-08-06 01:14:23 on dist-test-slave-7nm7"
server_key: "6fb217cf20d2f0d65ffa479ae4d39709"
server_key_iv: "7e006674bda7dc7d4ea4304c3387d28b"
server_key_version: "encryptionkey@0"
I20260806 01:14:23.665316 2528 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:14:23.669245 2549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:23.670799 2528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:14:23.670966 2528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "af5eca9b3ee1465084d78325c7932858"
format_stamp: "Formatted at 2026-08-06 01:14:23 on dist-test-slave-7nm7"
server_key: "6fb217cf20d2f0d65ffa479ae4d39709"
server_key_iv: "7e006674bda7dc7d4ea4304c3387d28b"
server_key_version: "encryptionkey@0"
I20260806 01:14:23.671157 2528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:23.693926 2528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:23.700825 2528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:23.701185 2528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:23.723203 2528 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:33351
I20260806 01:14:23.723241 2629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:33351 every 8 connection(s)
I20260806 01:14:23.724639 2528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:14:23.729306 2630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:23.730778 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 2528
I20260806 01:14:23.730993 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:14:23.731667 26546 external_mini_cluster.cc:1428] Setting key 45983de50af8dafc75d06db0cef9bd23
I20260806 01:14:23.741498 2630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858: Bootstrap starting.
I20260806 01:14:23.745595 2630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858: Neither blocks nor log segments found. Creating new log.
Aug 06 01:14:23 dist-test-slave-7nm7 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:23.746865 2630 log.cc:824] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:23.751024 2630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858: No bootstrap required, opened a new log
I20260806 01:14:23.758481 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:23.733814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37130 (local address 127.25.236.190:33351)
0806 01:14:23.734641 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:23.734662 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:23.734707 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:14:23.736377 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:23.736397 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:23.736764 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:23.737062 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:23.738878 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:23.740280 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:23.742036 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:23.742797 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:23.742938 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:23.748250 (+ 5312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:23.748297 (+ 47us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:23.748327 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:23.748462 (+ 135us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:23.753377 (+ 4915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:23.754172 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:23.754180 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:23.754190 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:23.754434 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:23.754853 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:23.754858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:23.754862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:23.755604 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:23.755923 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:23.756532 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 01:14:23.757001 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":396,"threads_started":1}
I20260806 01:14:23.762044 2630 raft_consensus.cc:348] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af5eca9b3ee1465084d78325c7932858" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33351 } }
I20260806 01:14:23.762485 2630 raft_consensus.cc:374] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:23.762562 2630 raft_consensus.cc:729] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af5eca9b3ee1465084d78325c7932858, State: Initialized, Role: FOLLOWER
I20260806 01:14:23.763067 2630 consensus_queue.cc:260] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af5eca9b3ee1465084d78325c7932858" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33351 } }
I20260806 01:14:23.763324 2630 raft_consensus.cc:388] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:14:23.763473 2630 raft_consensus.cc:482] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:14:23.763649 2630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:23.766181 2630 raft_consensus.cc:504] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af5eca9b3ee1465084d78325c7932858" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33351 } }
I20260806 01:14:23.766778 2630 leader_election.cc:302] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af5eca9b3ee1465084d78325c7932858; no voters:
I20260806 01:14:23.767328 2630 leader_election.cc:288] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:14:23.767472 2636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:23.768193 2636 raft_consensus.cc:686] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [term 1 LEADER]: Becoming Leader. State: Replica: af5eca9b3ee1465084d78325c7932858, State: Running, Role: LEADER
I20260806 01:14:23.768740 2636 consensus_queue.cc:237] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af5eca9b3ee1465084d78325c7932858" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33351 } }
I20260806 01:14:23.769682 2630 sys_catalog.cc:564] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:23.771843 2640 sys_catalog.cc:455] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af5eca9b3ee1465084d78325c7932858. Latest consensus state: current_term: 1 leader_uuid: "af5eca9b3ee1465084d78325c7932858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af5eca9b3ee1465084d78325c7932858" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33351 } } }
I20260806 01:14:23.772154 2638 sys_catalog.cc:455] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af5eca9b3ee1465084d78325c7932858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af5eca9b3ee1465084d78325c7932858" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33351 } } }
I20260806 01:14:23.772399 2640 sys_catalog.cc:458] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:23.772471 2638 sys_catalog.cc:458] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:23.774111 2643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:23.779407 2643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:23.789018 2643 catalog_manager.cc:1347] Generated new cluster ID: 510e72da22e44f429815d76eb5c87065
I20260806 01:14:23.789165 2643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:23.828233 2643 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:14:23.829596 2643 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:23.836777 2643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af5eca9b3ee1465084d78325c7932858: Generated new TSK 0
I20260806 01:14:23.837831 2643 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:14:23.909570 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33351
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:38191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:24.260605 2670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:24.261229 2670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:24.261474 2670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:24.269737 2670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:24.269881 2670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:24.270099 2670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:14:24.280465 2670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:38191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:24.281781 2670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:24.283186 2670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:24.296731 2678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:24.296924 2680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:24.297417 2677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:24.297837 2670 server_base.cc:1034] running on GCE node
I20260806 01:14:24.298593 2670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:24.299757 2670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:24.300988 2670 hybrid_clock.cc:648] HybridClock initialized: now 1785978864300922 us; error 61 us; skew 500 ppm
I20260806 01:14:24.301357 2670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:24 dist-test-slave-7nm7 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978864, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:24.305130 2670 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:14:24.306653 2670 webserver.cc:460] Webserver started at http://127.25.236.129:33193/ using document root <none> and password file <none>
I20260806 01:14:24.307292 2670 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:24.307394 2670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:24.307650 2670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:24.310595 2670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "582159fa5c3840ed9b73bb8e3ad10528"
format_stamp: "Formatted at 2026-08-06 01:14:24 on dist-test-slave-7nm7"
server_key: "33ca2d2e5a3fa5247a30cf0a4d8ee90e"
server_key_iv: "289204421a1cfb0e1d8e410a49224321"
server_key_version: "encryptionkey@0"
I20260806 01:14:24.311398 2670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "582159fa5c3840ed9b73bb8e3ad10528"
format_stamp: "Formatted at 2026-08-06 01:14:24 on dist-test-slave-7nm7"
server_key: "33ca2d2e5a3fa5247a30cf0a4d8ee90e"
server_key_iv: "289204421a1cfb0e1d8e410a49224321"
server_key_version: "encryptionkey@0"
I20260806 01:14:24.316759 2670 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:14:24.320349 2691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:24.321769 2670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 01:14:24.321975 2670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "582159fa5c3840ed9b73bb8e3ad10528"
format_stamp: "Formatted at 2026-08-06 01:14:24 on dist-test-slave-7nm7"
server_key: "33ca2d2e5a3fa5247a30cf0a4d8ee90e"
server_key_iv: "289204421a1cfb0e1d8e410a49224321"
server_key_version: "encryptionkey@0"
I20260806 01:14:24.322230 2670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:24.336759 2670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:24.341058 2670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:24.341361 2670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:24.342386 2670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:24.344018 2670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:24.344255 2670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:24.344429 2670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:24.344513 2670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:24.383683 2670 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:43347
I20260806 01:14:24.383690 2861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:43347 every 8 connection(s)
I20260806 01:14:24.385134 2670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:14:24.385886 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 2670
I20260806 01:14:24.386157 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:14:24.387018 26546 external_mini_cluster.cc:1428] Setting key 19e0070470158f0e501ae52067a4c324
Aug 06 01:14:24 dist-test-slave-7nm7 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978864, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:24.412817 2868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:24.388748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:58069 (local address 127.25.236.190:33351)
0806 01:14:24.389223 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:24.389229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:24.391046 (+ 1817us) server_negotiation.cc:407] Connection header received
0806 01:14:24.394038 (+ 2992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:24.394047 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:24.394272 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:24.394405 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:24.396617 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:24.397990 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:24.400335 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:24.401038 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:24.401194 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:24.405152 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:24.405161 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:24.405168 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:24.405214 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:24.408982 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:24.409790 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:24.409799 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:24.409805 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:24.409960 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:24.410564 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:24.410572 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:24.410578 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:24.410992 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:24.411246 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:24.412020 (+ 774us) server_negotiation.cc:299] Negotiation successful
0806 01:14:24.412310 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":255,"threads_started":1}
I20260806 01:14:24.413380 2869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:24.389789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:33351 (local address 127.25.236.129:58069)
0806 01:14:24.390840 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:24.390907 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:24.393693 (+ 2786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:24.394692 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:24.394706 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:24.395534 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:24.396394 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:24.396423 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:24.398178 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:24.398186 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:24.400187 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:24.400201 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:24.401488 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:24.401497 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:24.401859 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:24.402705 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:24.402730 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:24.404912 (+ 2182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:24.409195 (+ 4283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:24.409201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:24.409214 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:24.409642 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:24.410082 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:24.410085 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:24.410087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:24.410425 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:24.411178 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:24.411186 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:24.411564 (+ 378us) client_negotiation.cc:769] Sending connection context
0806 01:14:24.411784 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 01:14:24.412254 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":500,"threads_started":1}
I20260806 01:14:24.415612 2863 heartbeater.cc:347] Connected to a master server at 127.25.236.190:33351
I20260806 01:14:24.415937 2863 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:24.416694 2863 heartbeater.cc:511] Master 127.25.236.190:33351 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
I20260806 01:14:24.419005 2573 ts_manager.cc:194] Registered new tserver with Master: 582159fa5c3840ed9b73bb8e3ad10528 (127.25.236.129:43347)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
I20260806 01:14:24.420581 2573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:58069
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:14:24.449709 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33351
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:38191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:24.817662 2874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:24.817938 2874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:24.818037 2874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:24.825281 2874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:24.825403 2874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:24.825519 2874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:14:24.835716 2874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:38191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:24.837090 2874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:24.838464 2874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:24.852686 2880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:24.852612 2881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:24.852643 2886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:24.853855 2874 server_base.cc:1034] running on GCE node
I20260806 01:14:24.854496 2874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:24.855386 2874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:24.856596 2874 hybrid_clock.cc:648] HybridClock initialized: now 1785978864856550 us; error 30 us; skew 500 ppm
I20260806 01:14:24.856904 2874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:24 dist-test-slave-7nm7 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978864, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:24.860376 2874 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:14:24.861994 2874 webserver.cc:460] Webserver started at http://127.25.236.130:32829/ using document root <none> and password file <none>
I20260806 01:14:24.862782 2874 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:24.862941 2874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:24.863269 2874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:24.866575 2874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "1df31858c6fc4a7f82ba6f5ebd409f26"
format_stamp: "Formatted at 2026-08-06 01:14:24 on dist-test-slave-7nm7"
server_key: "dbb365190b150c976cf81ffa2606e95f"
server_key_iv: "7559c34ff109dd7736f86c7c18494b4d"
server_key_version: "encryptionkey@0"
I20260806 01:14:24.867470 2874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "1df31858c6fc4a7f82ba6f5ebd409f26"
format_stamp: "Formatted at 2026-08-06 01:14:24 on dist-test-slave-7nm7"
server_key: "dbb365190b150c976cf81ffa2606e95f"
server_key_iv: "7559c34ff109dd7736f86c7c18494b4d"
server_key_version: "encryptionkey@0"
I20260806 01:14:24.873230 2874 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:14:24.877497 2894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:24.879061 2874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:14:24.879432 2874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "1df31858c6fc4a7f82ba6f5ebd409f26"
format_stamp: "Formatted at 2026-08-06 01:14:24 on dist-test-slave-7nm7"
server_key: "dbb365190b150c976cf81ffa2606e95f"
server_key_iv: "7559c34ff109dd7736f86c7c18494b4d"
server_key_version: "encryptionkey@0"
I20260806 01:14:24.879770 2874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:24.904035 2874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:24.909248 2874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:24.909578 2874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:24.910763 2874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:24.912147 2874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:24.912264 2874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:24.912415 2874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:24.912489 2874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:24.955658 2874 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:42715
I20260806 01:14:24.955719 3066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:42715 every 8 connection(s)
I20260806 01:14:24.957159 2874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:14:24.959599 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 2874
I20260806 01:14:24.959792 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:14:24.960407 26546 external_mini_cluster.cc:1428] Setting key f1994f33213f26bd46d235d00c2cc375
Aug 06 01:14:24 dist-test-slave-7nm7 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978864, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:24.986263 3069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:24.960126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:53475 (local address 127.25.236.190:33351)
0806 01:14:24.960633 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:24.960641 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:24.961825 (+ 1184us) server_negotiation.cc:407] Connection header received
0806 01:14:24.964678 (+ 2853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:24.964682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:24.964762 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:24.964864 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:24.967260 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:24.968371 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:24.972046 (+ 3675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:24.972578 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:24.972692 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:24.977506 (+ 4814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:24.977515 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:24.977523 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:24.977577 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:24.982040 (+ 4463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:24.983012 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:24.983021 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:24.983027 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:24.983204 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:24.983713 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:24.983720 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:24.983726 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:24.984188 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:24.984349 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:24.985431 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0806 01:14:24.985693 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":261,"threads_started":1}
I20260806 01:14:24.986826 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:24.960691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:33351 (local address 127.25.236.130:53475)
0806 01:14:24.961666 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:24.961707 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:24.964334 (+ 2627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:24.965163 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:24.965179 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:24.965998 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:24.966895 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:24.966924 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:24.968555 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:24.968563 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:24.971819 (+ 3256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:24.971840 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:24.972841 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:24.972851 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:24.973228 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:24.974107 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:24.974146 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:24.977191 (+ 3045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:24.982330 (+ 5139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:24.982338 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:24.982353 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:24.982853 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:24.983364 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:24.983368 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:24.983370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:24.983583 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:24.984366 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:24.984382 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:24.984786 (+ 404us) client_negotiation.cc:769] Sending connection context
0806 01:14:24.985056 (+ 270us) client_negotiation.cc:240] Negotiation successful
0806 01:14:24.985532 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":410,"threads_started":1}
I20260806 01:14:24.988809 3067 heartbeater.cc:347] Connected to a master server at 127.25.236.190:33351
I20260806 01:14:24.989145 3067 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:24.990347 3067 heartbeater.cc:511] Master 127.25.236.190:33351 requested a full tablet report, sending...
I20260806 01:14:24.992206 2573 ts_manager.cc:194] Registered new tserver with Master: 1df31858c6fc4a7f82ba6f5ebd409f26 (127.25.236.130:42715)
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
I20260806 01:14:24.993253 2573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:53475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:14:25.033234 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33351
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:38191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:25.408282 3077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:25.408593 3077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:25.408761 3077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:25.417228 3077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:25.417366 3077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:25.417569 3077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:14:25.424314 2863 heartbeater.cc:503] Master 127.25.236.190:33351 was elected leader, sending a full tablet report...
I20260806 01:14:25.431121 3077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:38191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:25.432920 3077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:25.435117 3077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:25.448658 3084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:25.450534 3088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:25.450870 3085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:25.452267 3077 server_base.cc:1034] running on GCE node
I20260806 01:14:25.452946 3077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:25.454022 3077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:25.455319 3077 hybrid_clock.cc:648] HybridClock initialized: now 1785978865455255 us; error 56 us; skew 500 ppm
I20260806 01:14:25.455680 3077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:25.459143 3077 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:14:25.460762 3077 webserver.cc:460] Webserver started at http://127.25.236.131:43077/ using document root <none> and password file <none>
I20260806 01:14:25.461613 3077 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:25.461720 3077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:25.461982 3077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:25.465116 3077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "487253f971804d97a8c28dbf706dc176"
format_stamp: "Formatted at 2026-08-06 01:14:25 on dist-test-slave-7nm7"
server_key: "707424dae38757f179fe6e0f5a64c9e3"
server_key_iv: "6e364f4eed01d97738bb64faab47a019"
server_key_version: "encryptionkey@0"
I20260806 01:14:25.465862 3077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "487253f971804d97a8c28dbf706dc176"
format_stamp: "Formatted at 2026-08-06 01:14:25 on dist-test-slave-7nm7"
server_key: "707424dae38757f179fe6e0f5a64c9e3"
server_key_iv: "6e364f4eed01d97738bb64faab47a019"
server_key_version: "encryptionkey@0"
I20260806 01:14:25.471122 3077 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 01:14:25.474582 3099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:25.476022 3077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:14:25.476214 3077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "487253f971804d97a8c28dbf706dc176"
format_stamp: "Formatted at 2026-08-06 01:14:25 on dist-test-slave-7nm7"
server_key: "707424dae38757f179fe6e0f5a64c9e3"
server_key_iv: "6e364f4eed01d97738bb64faab47a019"
server_key_version: "encryptionkey@0"
I20260806 01:14:25.476392 3077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:25.494767 3077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:25.499205 3077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:25.499460 3077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:25.500720 3077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:25.502761 3077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:25.502863 3077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:25.502956 3077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:25.503002 3077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:25.540251 3077 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:36225
I20260806 01:14:25.540345 3273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:36225 every 8 connection(s)
I20260806 01:14:25.541580 3077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:14:25.543215 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 3077
I20260806 01:14:25.543396 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:14:25.543941 26546 external_mini_cluster.cc:1428] Setting key 5a5e0ef0c9ad7ddb53d44425704ee3c9
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:25.563807 3277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:25.544628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:42073 (local address 127.25.236.190:33351)
0806 01:14:25.545174 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:25.545178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:25.547990 (+ 2812us) server_negotiation.cc:407] Connection header received
0806 01:14:25.548040 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:25.548043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:25.548103 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:25.548195 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:25.549715 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.550668 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.552250 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.552697 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.552797 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:25.556487 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:25.556492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:25.556495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:25.556526 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:25.560635 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.561482 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.561487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.561499 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.561657 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.562114 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.562118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.562120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.562495 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:25.562632 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:25.563333 (+ 701us) server_negotiation.cc:299] Negotiation successful
0806 01:14:25.563517 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":277,"threads_started":1}
I20260806 01:14:25.564755 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:25.545089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:33351 (local address 127.25.236.131:42073)
0806 01:14:25.545959 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:25.546014 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:25.547732 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:25.548454 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:25.548462 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:25.548968 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:25.549535 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:25.549552 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.550837 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.550841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:25.552085 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:25.552098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.552916 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.552921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:25.553119 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:25.553808 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:25.553829 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:25.556235 (+ 2406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:25.560848 (+ 4613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:25.560855 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:25.560869 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:25.561333 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:25.561780 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:25.561784 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:25.561786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:25.561982 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:25.562633 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:25.562639 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:25.562963 (+ 324us) client_negotiation.cc:769] Sending connection context
0806 01:14:25.563206 (+ 243us) client_negotiation.cc:240] Negotiation successful
0806 01:14:25.563610 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":380,"threads_started":1}
I20260806 01:14:25.566314 3274 heartbeater.cc:347] Connected to a master server at 127.25.236.190:33351
I20260806 01:14:25.566677 3274 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:25.567503 3274 heartbeater.cc:511] Master 127.25.236.190:33351 requested a full tablet report, sending...
I20260806 01:14:25.569113 2573 ts_manager.cc:194] Registered new tserver with Master: 487253f971804d97a8c28dbf706dc176 (127.25.236.131:36225)
I20260806 01:14:25.570336 2573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:42073
I20260806 01:14:25.572582 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/test-admin.keytab.
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:25.605509 26546 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 01:14:25.611183 3288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:25.608295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55554 (local address 127.25.236.129:43347)
0806 01:14:25.609050 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:25.609057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:25.609090 (+ 33us) server_negotiation.cc:407] Connection header received
0806 01:14:25.609160 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:25.609167 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:25.609385 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:25.609565 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:25.610719 (+ 1154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55554: BlockingRecv error: recv got EOF from 127.0.0.1:55554 (error 108)
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":303,"threads_started":1}
W20260806 01:14:25.611289 26546 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/krb5cc' of type FILE
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:25.613036 26546 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:25.614698 26546 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:14:25.630806 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:25.616478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55562 (local address 127.25.236.129:43347)
0806 01:14:25.616698 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:25.616705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:25.616875 (+ 170us) server_negotiation.cc:407] Connection header received
0806 01:14:25.617072 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:25.617078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:25.617153 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:25.617258 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:25.618323 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.619307 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.620524 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.621086 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.621230 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:25.624960 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:25.624971 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:25.624986 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:25.625025 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:25.628085 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.628772 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.628779 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.628781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.628962 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.629322 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.629325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.629327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.629663 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:25.629867 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:25.630196 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 01:14:25.630442 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260806 01:14:25.636838 3288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:25.635233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55576 (local address 127.25.236.129:43347)
0806 01:14:25.635464 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:25.635469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:25.635659 (+ 190us) server_negotiation.cc:407] Connection header received
0806 01:14:25.635773 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:25.635777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:25.635842 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:25.635917 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:25.636626 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55576: BlockingRecv error: recv got EOF from 127.0.0.1:55576 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260806 01:14:25.637008 26546 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/krb5cc' of type FILE
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:25.638536 26546 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:25.639977 26546 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:14:25.654315 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:25.641184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55584 (local address 127.25.236.129:43347)
0806 01:14:25.641390 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:25.641397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:25.641621 (+ 224us) server_negotiation.cc:407] Connection header received
0806 01:14:25.641731 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:25.641735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:25.641796 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:25.641873 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:25.642881 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.643570 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.644746 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.645164 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.645332 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:25.649163 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:25.649175 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:25.649180 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:25.649216 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:25.651733 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.652368 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.652375 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.652379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.652505 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.652883 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.652888 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.652892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.653171 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:25.653442 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:25.653758 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 01:14:25.653965 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 01:14:25.658434 3288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:25.656797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55592 (local address 127.25.236.129:43347)
0806 01:14:25.657017 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:25.657022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:25.657256 (+ 234us) server_negotiation.cc:407] Connection header received
0806 01:14:25.657367 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:25.657370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:25.657437 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:25.657513 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:25.658158 (+ 645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55592: BlockingRecv error: recv got EOF from 127.0.0.1:55592 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260806 01:14:25.658854 26546 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978805316623-26546-0/krb5kdc/krb5cc)
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:25.660302 26546 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:25.661717 26546 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:14:25.678444 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:25.663288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55598 (local address 127.25.236.129:43347)
0806 01:14:25.663510 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:25.663516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:25.663728 (+ 212us) server_negotiation.cc:407] Connection header received
0806 01:14:25.663911 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:25.663916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:25.663970 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:25.664068 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:25.664961 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.665691 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.667016 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.667443 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.667583 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:25.672417 (+ 4834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:25.672422 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:25.672426 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:25.672458 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:25.675157 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.675986 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.675990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.675992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.676154 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.676606 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.676610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.676612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.676916 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:25.677179 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:25.677650 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 01:14:25.677943 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:14:25.680394 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 2670
I20260806 01:14:25.693928 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:25.681212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55610 (local address 127.25.236.129:43347)
0806 01:14:25.681420 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:25.681429 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:25.681931 (+ 502us) server_negotiation.cc:407] Connection header received
0806 01:14:25.682363 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:25.682371 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:25.682473 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:25.682577 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:25.683686 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.684450 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.685761 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.686170 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.686345 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:25.688471 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:25.688476 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:25.688479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:25.688510 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:25.691148 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.691797 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.691801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.691804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.691950 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.692367 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.692371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.692373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.692632 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:25.692917 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:25.693267 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 01:14:25.693446 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 01:14:25.694669 2856 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:25.788941 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 2670
I20260806 01:14:25.818321 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 2874
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:14:25.836596 3303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:25.819311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58066 (local address 127.25.236.130:42715)
0806 01:14:25.819889 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:25.819896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:25.819921 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:14:25.819979 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:25.819986 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:25.820193 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:25.820319 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:25.821457 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.822793 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.824239 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.824907 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.825021 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:25.829681 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:25.829693 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:25.829710 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:25.829757 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:25.833236 (+ 3479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.834084 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.834091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.834094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.834490 (+ 396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.834909 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.834915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.834920 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.835317 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:25.835645 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:25.835983 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 01:14:25.836266 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":305,"threads_started":1}
I20260806 01:14:25.837940 3061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:25.923200 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 2874
I20260806 01:14:25.948869 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 3077
Aug 06 01:14:25 dist-test-slave-7nm7 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:14:25.964345 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:25.950080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45650 (local address 127.25.236.131:36225)
0806 01:14:25.950836 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:25.950844 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:25.950873 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:14:25.950933 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:25.950940 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:25.951169 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:25.951295 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:25.952320 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.953336 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.954606 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:25.955173 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:25.955278 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:25.959322 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:25.959333 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:25.959346 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:25.959383 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:25.961939 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.962649 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.962656 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.962658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.962778 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:25.963092 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:25.963095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:25.963097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:25.963411 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:25.963628 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:25.963871 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 01:14:25.964064 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":434,"threads_started":1}
I20260806 01:14:25.965618 3264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:26.050550 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 3077
I20260806 01:14:26.074432 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 2528
I20260806 01:14:26.076325 2623 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:26.185395 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 2528
2026-08-06T01:14:26Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4643 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:14:26 dist-test-slave-7nm7 krb5kdc[3310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:14:26 dist-test-slave-7nm7 krb5kdc[3310](info): set up 2 sockets
Aug 06 01:14:26 dist-test-slave-7nm7 krb5kdc[3310](info): commencing operation
krb5kdc: starting...
W20260806 01:14:27.576192 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:14:27 dist-test-slave-7nm7 krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:14:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:14:27Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.190.keytab.
Entry for principal oryx/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.190.keytab.
I20260806 01:14:27.868410 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:35167
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.190.keytab
--principal=oryx/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:35167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:28.250473 3329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:28.250777 3329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:28.250954 3329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:28.258070 3329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:28.258165 3329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:28.258282 3329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:28.258399 3329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:28.258481 3329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:14:28.268944 3329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:35167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.190.keytab
--principal=oryx/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:35167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:28.270397 3329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:28.271766 3329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:28.283951 3337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:28.283970 3344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:28.283907 3340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:28.285158 3329 server_base.cc:1034] running on GCE node
I20260806 01:14:28.285934 3329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:28.287596 3329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:28.288831 3329 hybrid_clock.cc:648] HybridClock initialized: now 1785978868288765 us; error 56 us; skew 500 ppm
I20260806 01:14:28.289183 3329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:28 dist-test-slave-7nm7 krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978868, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:28.292699 3329 init.cc:377] Logged in from keytab as oryx/127.25.236.190@KRBTEST.COM (short username oryx)
I20260806 01:14:28.294598 3329 webserver.cc:460] Webserver started at http://127.25.236.190:39489/ using document root <none> and password file <none>
I20260806 01:14:28.295307 3329 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:28.295444 3329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:28.295778 3329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:28.299098 3329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "d0dac0217f664480abd32c52e2e36380"
format_stamp: "Formatted at 2026-08-06 01:14:28 on dist-test-slave-7nm7"
server_key: "80805464909c6c932c5cd4d142bae73b"
server_key_iv: "89139da53e0bbbba6537a450b08ae38c"
server_key_version: "encryptionkey@0"
I20260806 01:14:28.299906 3329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "d0dac0217f664480abd32c52e2e36380"
format_stamp: "Formatted at 2026-08-06 01:14:28 on dist-test-slave-7nm7"
server_key: "80805464909c6c932c5cd4d142bae73b"
server_key_iv: "89139da53e0bbbba6537a450b08ae38c"
server_key_version: "encryptionkey@0"
I20260806 01:14:28.305667 3329 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:14:28.309489 3356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:28.311161 3329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:14:28.311379 3329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "d0dac0217f664480abd32c52e2e36380"
format_stamp: "Formatted at 2026-08-06 01:14:28 on dist-test-slave-7nm7"
server_key: "80805464909c6c932c5cd4d142bae73b"
server_key_iv: "89139da53e0bbbba6537a450b08ae38c"
server_key_version: "encryptionkey@0"
I20260806 01:14:28.311602 3329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:28.330435 3329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:28.335858 3329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:28.336191 3329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:28.357708 3329 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:35167
I20260806 01:14:28.357757 3442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:35167 every 8 connection(s)
I20260806 01:14:28.359607 3329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:14:28.363947 3443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:28.369349 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 3329
I20260806 01:14:28.369573 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:14:28.370402 26546 external_mini_cluster.cc:1428] Setting key aaaa7e4ebab646b90676fefb6890cd11
I20260806 01:14:28.376854 3443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380: Bootstrap starting.
I20260806 01:14:28.381131 3443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:28.382457 3443 log.cc:824] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:28.386018 3443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380: No bootstrap required, opened a new log
Aug 06 01:14:28 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:28.394634 3443 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } }
I20260806 01:14:28.395043 3443 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:28.395331 3443 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0dac0217f664480abd32c52e2e36380, State: Initialized, Role: FOLLOWER
I20260806 01:14:28.396081 3443 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } }
I20260806 01:14:28.396473 3443 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:14:28.396647 3443 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:14:28.396864 3443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:28.397024 3446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:28.372712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47010 (local address 127.25.236.190:35167)
0806 01:14:28.373901 (+ 1189us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:28.373913 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:28.373961 (+ 48us) server_negotiation.cc:407] Connection header received
0806 01:14:28.375968 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:28.375991 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:28.376377 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:28.376705 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:28.379052 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:28.380756 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:28.382633 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:28.383719 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:28.383916 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:28.388690 (+ 4774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:28.388699 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:28.388712 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:28.388756 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:28.392162 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:28.392880 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:28.392890 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:28.392896 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:28.393072 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:28.393597 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:28.393601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:28.393603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:28.394399 (+ 796us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:28.394766 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:28.395356 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 01:14:28.395840 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":956,"thread_start_us":733,"threads_started":1}
I20260806 01:14:28.399655 3443 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } }
I20260806 01:14:28.400198 3443 leader_election.cc:302] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0dac0217f664480abd32c52e2e36380; no voters:
I20260806 01:14:28.400831 3443 leader_election.cc:288] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:14:28.400988 3452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:28.401851 3452 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [term 1 LEADER]: Becoming Leader. State: Replica: d0dac0217f664480abd32c52e2e36380, State: Running, Role: LEADER
I20260806 01:14:28.402602 3452 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } }
I20260806 01:14:28.403128 3443 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:28.405581 3453 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } }
I20260806 01:14:28.405946 3453 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:28.405905 3454 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0dac0217f664480abd32c52e2e36380. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } }
I20260806 01:14:28.407265 3454 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:28.408448 3460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:28.413041 3460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:28.422747 3460 catalog_manager.cc:1347] Generated new cluster ID: 48e3156596764ed7a8f5e71c7c835421
I20260806 01:14:28.422870 3460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:28.438047 3460 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:14:28.440187 3460 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:28.450675 3460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380: Generated new TSK 0
I20260806 01:14:28.451699 3460 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab.
Entry for principal oryx/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab.
I20260806 01:14:28.526687 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:35167
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab
--principal=oryx/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:28.910276 3481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:28.910590 3481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:28.910785 3481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:28.919452 3481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:28.919582 3481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:28.919824 3481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:14:28.931049 3481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab
--principal=oryx/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:35167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:28.932448 3481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:28.933951 3481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:28.948975 3489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:28.949029 3488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:28.949110 3491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:28.950901 3481 server_base.cc:1034] running on GCE node
I20260806 01:14:28.951586 3481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:28.952584 3481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:28.953814 3481 hybrid_clock.cc:648] HybridClock initialized: now 1785978868953749 us; error 52 us; skew 500 ppm
I20260806 01:14:28.954200 3481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:28 dist-test-slave-7nm7 krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978868, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:28.962857 3481 init.cc:377] Logged in from keytab as oryx/127.25.236.129@KRBTEST.COM (short username oryx)
I20260806 01:14:28.964908 3481 webserver.cc:460] Webserver started at http://127.25.236.129:46879/ using document root <none> and password file <none>
I20260806 01:14:28.965700 3481 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:28.965812 3481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:28.966212 3481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:28.969437 3481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "c1def5a943184a4aabe01c99379d51d2"
format_stamp: "Formatted at 2026-08-06 01:14:28 on dist-test-slave-7nm7"
server_key: "a1d76fcda5610d16037fb9564577e0ef"
server_key_iv: "af138e36d6a9f2abc06e1b5c29010e76"
server_key_version: "encryptionkey@0"
I20260806 01:14:28.970393 3481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "c1def5a943184a4aabe01c99379d51d2"
format_stamp: "Formatted at 2026-08-06 01:14:28 on dist-test-slave-7nm7"
server_key: "a1d76fcda5610d16037fb9564577e0ef"
server_key_iv: "af138e36d6a9f2abc06e1b5c29010e76"
server_key_version: "encryptionkey@0"
I20260806 01:14:28.976606 3481 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 01:14:28.981423 3500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:28.983283 3481 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 01:14:28.983496 3481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "c1def5a943184a4aabe01c99379d51d2"
format_stamp: "Formatted at 2026-08-06 01:14:28 on dist-test-slave-7nm7"
server_key: "a1d76fcda5610d16037fb9564577e0ef"
server_key_iv: "af138e36d6a9f2abc06e1b5c29010e76"
server_key_version: "encryptionkey@0"
I20260806 01:14:28.983757 3481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:29.002888 3481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:29.007680 3481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:29.008008 3481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:29.009207 3481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:29.010661 3481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:29.010768 3481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:29.010937 3481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:29.011020 3481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:29.049930 3481 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:44437
I20260806 01:14:29.050024 3696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:44437 every 8 connection(s)
I20260806 01:14:29.051319 3481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:14:29.058794 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 3481
I20260806 01:14:29.059156 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:14:29.060101 26546 external_mini_cluster.cc:1428] Setting key 8bfd45e78f4b273c2955937c6f5dcac5
Aug 06 01:14:29 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978868, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:29.075472 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:29.053991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:41907 (local address 127.25.236.190:35167)
0806 01:14:29.054596 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:29.054602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:29.055466 (+ 864us) server_negotiation.cc:407] Connection header received
0806 01:14:29.057389 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:29.057395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:29.057477 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:29.057582 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:29.059582 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:29.060695 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:29.063263 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:29.063929 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:29.064092 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:29.067860 (+ 3768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:29.067871 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:29.067878 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:29.067925 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:29.071574 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:29.072375 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:29.072383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:29.072390 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:29.072552 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:29.073045 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:29.073052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:29.073058 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:29.073754 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:29.073885 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:29.074977 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0806 01:14:29.075185 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":341,"threads_started":1}
I20260806 01:14:29.077287 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:29.054503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.129:41907)
0806 01:14:29.055301 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:29.055339 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:29.056986 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:29.057807 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:29.057816 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:29.058651 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:29.059370 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:29.059392 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:29.060922 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:29.060928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:29.063035 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:29.063049 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:29.064208 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:29.064214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:29.064434 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:29.065304 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:29.065332 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:29.067619 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:29.071741 (+ 4122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:29.071749 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:29.071762 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:29.072225 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:29.072711 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:29.072714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:29.072717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:29.072905 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:29.073916 (+ 1011us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:29.073926 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:29.074477 (+ 551us) client_negotiation.cc:769] Sending connection context
0806 01:14:29.074855 (+ 378us) client_negotiation.cc:240] Negotiation successful
0806 01:14:29.075500 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":306,"threads_started":1}
I20260806 01:14:29.078994 3698 heartbeater.cc:347] Connected to a master server at 127.25.236.190:35167
I20260806 01:14:29.079321 3698 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:29.080286 3698 heartbeater.cc:511] Master 127.25.236.190:35167 requested a full tablet report, sending...
I20260806 01:14:29.082746 3384 ts_manager.cc:194] Registered new tserver with Master: c1def5a943184a4aabe01c99379d51d2 (127.25.236.129:44437)
I20260806 01:14:29.084208 3384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:41907
WARNING: no policy specified for oryx/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab.
Entry for principal oryx/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab.
I20260806 01:14:29.125900 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:35167
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab
--principal=oryx/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:29.513865 3710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:29.514232 3710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:29.514446 3710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:29.525092 3710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:29.525326 3710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:29.525522 3710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:14:29.536827 3710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab
--principal=oryx/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:35167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:29.538218 3710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:29.539621 3710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:29.554075 3716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:29.554075 3717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:29.554415 3720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:29.555023 3710 server_base.cc:1034] running on GCE node
I20260806 01:14:29.555591 3710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:29.556531 3710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:29.557739 3710 hybrid_clock.cc:648] HybridClock initialized: now 1785978869557700 us; error 46 us; skew 500 ppm
I20260806 01:14:29.558096 3710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:29 dist-test-slave-7nm7 krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978869, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:29.562016 3710 init.cc:377] Logged in from keytab as oryx/127.25.236.130@KRBTEST.COM (short username oryx)
I20260806 01:14:29.563956 3710 webserver.cc:460] Webserver started at http://127.25.236.130:33525/ using document root <none> and password file <none>
I20260806 01:14:29.564675 3710 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:29.564847 3710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:29.565171 3710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:29.568184 3710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "e3691097671c4d49b2b6f1b7846272bf"
format_stamp: "Formatted at 2026-08-06 01:14:29 on dist-test-slave-7nm7"
server_key: "a12c7d6a0722da8da26c7d7743793f1b"
server_key_iv: "4d8a20c91c15adae19df18c82788fae2"
server_key_version: "encryptionkey@0"
I20260806 01:14:29.568974 3710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "e3691097671c4d49b2b6f1b7846272bf"
format_stamp: "Formatted at 2026-08-06 01:14:29 on dist-test-slave-7nm7"
server_key: "a12c7d6a0722da8da26c7d7743793f1b"
server_key_iv: "4d8a20c91c15adae19df18c82788fae2"
server_key_version: "encryptionkey@0"
I20260806 01:14:29.574216 3710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:14:29.577601 3729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:29.579070 3710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 01:14:29.579254 3710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "e3691097671c4d49b2b6f1b7846272bf"
format_stamp: "Formatted at 2026-08-06 01:14:29 on dist-test-slave-7nm7"
server_key: "a12c7d6a0722da8da26c7d7743793f1b"
server_key_iv: "4d8a20c91c15adae19df18c82788fae2"
server_key_version: "encryptionkey@0"
I20260806 01:14:29.579460 3710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:29.607311 3710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:29.611907 3710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:29.612239 3710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:29.613237 3710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:29.614748 3710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:29.614859 3710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:29.615010 3710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:29.615092 3710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:29.652187 3710 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:36927
I20260806 01:14:29.652230 3908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:36927 every 8 connection(s)
I20260806 01:14:29.653514 3710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:14:29.655793 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 3710
I20260806 01:14:29.655992 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:14:29.656476 26546 external_mini_cluster.cc:1428] Setting key 8b0657402d08f0a78846575d69531531
Aug 06 01:14:29 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978869, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:29.677738 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:29.656944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:35601 (local address 127.25.236.190:35167)
0806 01:14:29.657549 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:29.657555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:29.658654 (+ 1099us) server_negotiation.cc:407] Connection header received
0806 01:14:29.660943 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:29.660948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:29.661038 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:29.661172 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:29.663473 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:29.664217 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:29.665860 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:29.666346 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:29.666544 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:29.670956 (+ 4412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:29.670962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:29.670965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:29.671000 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:29.673746 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:29.675042 (+ 1296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:29.675046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:29.675049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:29.675210 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:29.675761 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:29.675765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:29.675767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:29.676045 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:29.676382 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:29.677285 (+ 903us) server_negotiation.cc:299] Negotiation successful
0806 01:14:29.677470 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":375,"threads_started":1}
I20260806 01:14:29.679471 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:29.657527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.130:35601)
0806 01:14:29.658504 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:29.658551 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:29.660666 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:29.661439 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:29.661447 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:29.662637 (+ 1190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:29.663268 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:29.663290 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:29.664395 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:29.664401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:29.665677 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:29.665696 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:29.666612 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:29.666623 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:29.666892 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:29.667716 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:29.667741 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:29.670672 (+ 2931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:29.673903 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:29.673915 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:29.673935 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:29.674878 (+ 943us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:29.675333 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:29.675341 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:29.675347 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:29.675635 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:29.676269 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:29.676280 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:29.676756 (+ 476us) client_negotiation.cc:769] Sending connection context
0806 01:14:29.677085 (+ 329us) client_negotiation.cc:240] Negotiation successful
0806 01:14:29.677753 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":371,"threads_started":1}
I20260806 01:14:29.681289 3912 heartbeater.cc:347] Connected to a master server at 127.25.236.190:35167
I20260806 01:14:29.681656 3912 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:29.682785 3912 heartbeater.cc:511] Master 127.25.236.190:35167 requested a full tablet report, sending...
I20260806 01:14:29.684782 3384 ts_manager.cc:194] Registered new tserver with Master: e3691097671c4d49b2b6f1b7846272bf (127.25.236.130:36927)
I20260806 01:14:29.686134 3384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:35601
WARNING: no policy specified for oryx/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab.
Entry for principal oryx/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab.
I20260806 01:14:29.725991 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:35167
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab
--principal=oryx/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:14:30.087497 3698 heartbeater.cc:503] Master 127.25.236.190:35167 was elected leader, sending a full tablet report...
W20260806 01:14:30.098486 3920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:30.098842 3920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:30.099021 3920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:30.107376 3920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:30.107549 3920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:30.107728 3920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:14:30.120720 3920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab
--principal=oryx/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:35167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:30.122537 3920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:30.123979 3920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:30.138693 3928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:30.138733 3935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:30.138693 3931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:30.140040 3920 server_base.cc:1034] running on GCE node
I20260806 01:14:30.140738 3920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:30.141851 3920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:30.143198 3920 hybrid_clock.cc:648] HybridClock initialized: now 1785978870143071 us; error 59 us; skew 500 ppm
I20260806 01:14:30.143563 3920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:30 dist-test-slave-7nm7 krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978870, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:30.146721 3920 init.cc:377] Logged in from keytab as oryx/127.25.236.131@KRBTEST.COM (short username oryx)
I20260806 01:14:30.148334 3920 webserver.cc:460] Webserver started at http://127.25.236.131:38181/ using document root <none> and password file <none>
I20260806 01:14:30.149042 3920 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:30.149201 3920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:30.149518 3920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:30.152498 3920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "7de709fd7b784db4997f104e2ca0f5a9"
format_stamp: "Formatted at 2026-08-06 01:14:30 on dist-test-slave-7nm7"
server_key: "1ebcd38274241ad86f6b14d6f6cbf919"
server_key_iv: "dcf630c6fd0c45cfa52754e779733c2e"
server_key_version: "encryptionkey@0"
I20260806 01:14:30.153388 3920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "7de709fd7b784db4997f104e2ca0f5a9"
format_stamp: "Formatted at 2026-08-06 01:14:30 on dist-test-slave-7nm7"
server_key: "1ebcd38274241ad86f6b14d6f6cbf919"
server_key_iv: "dcf630c6fd0c45cfa52754e779733c2e"
server_key_version: "encryptionkey@0"
I20260806 01:14:30.159127 3920 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 01:14:30.162813 3945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:30.164237 3920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 01:14:30.164443 3920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "7de709fd7b784db4997f104e2ca0f5a9"
format_stamp: "Formatted at 2026-08-06 01:14:30 on dist-test-slave-7nm7"
server_key: "1ebcd38274241ad86f6b14d6f6cbf919"
server_key_iv: "dcf630c6fd0c45cfa52754e779733c2e"
server_key_version: "encryptionkey@0"
I20260806 01:14:30.164669 3920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:30.190932 3920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:30.196060 3920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:30.196331 3920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:30.197239 3920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:30.198729 3920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:30.198804 3920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:30.198887 3920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:30.198932 3920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:30.234750 3920 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:41223
I20260806 01:14:30.234791 4110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:41223 every 8 connection(s)
I20260806 01:14:30.236150 3920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:14:30.239290 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 3920
I20260806 01:14:30.239463 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:14:30.240091 26546 external_mini_cluster.cc:1428] Setting key 3496f9a85e0e30f245413efcdce1d333
Aug 06 01:14:30 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978870, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:30.257153 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:30.238992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:53979 (local address 127.25.236.190:35167)
0806 01:14:30.239568 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:30.239573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:30.240616 (+ 1043us) server_negotiation.cc:407] Connection header received
0806 01:14:30.242739 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:30.242744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:30.242831 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:30.242944 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:30.244400 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.245295 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.246900 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.247310 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.247445 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:30.251099 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:30.251115 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:30.251120 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:30.251153 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:30.253584 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:30.254543 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:30.254549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:30.254552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:30.254703 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:30.255277 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:30.255284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:30.255286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:30.255695 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:30.255873 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:30.256691 (+ 818us) server_negotiation.cc:299] Negotiation successful
0806 01:14:30.256855 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":356,"threads_started":1}
I20260806 01:14:30.258183 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:30.239655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.131:53979)
0806 01:14:30.240453 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:30.240498 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:30.242482 (+ 1984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:30.243175 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:30.243183 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:30.243666 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:30.244216 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:30.244232 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.245477 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.245481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:30.246749 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:30.246765 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.247536 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.247543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:30.247761 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:30.248521 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:30.248544 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:30.250897 (+ 2353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:30.253754 (+ 2857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:30.253761 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:30.253772 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:30.254406 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:30.254843 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:30.254847 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:30.254849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:30.255097 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:30.255914 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:30.255924 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:30.256309 (+ 385us) client_negotiation.cc:769] Sending connection context
0806 01:14:30.256570 (+ 261us) client_negotiation.cc:240] Negotiation successful
0806 01:14:30.256956 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":350,"threads_started":1}
I20260806 01:14:30.259631 4112 heartbeater.cc:347] Connected to a master server at 127.25.236.190:35167
I20260806 01:14:30.259930 4112 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:30.260756 4112 heartbeater.cc:511] Master 127.25.236.190:35167 requested a full tablet report, sending...
I20260806 01:14:30.262389 3384 ts_manager.cc:194] Registered new tserver with Master: 7de709fd7b784db4997f104e2ca0f5a9 (127.25.236.131:41223)
I20260806 01:14:30.263212 3384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:53979
I20260806 01:14:30.268563 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.236.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.236.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.236.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.189.keytab.
Entry for principal oryx/127.25.236.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.189.keytab.
I20260806 01:14:30.334596 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.189:34529
--webserver_interface=127.25.236.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.189.keytab
--principal=oryx/127.25.236.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:35167,127.25.236.189:34529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:14:30.689117 3912 heartbeater.cc:503] Master 127.25.236.190:35167 was elected leader, sending a full tablet report...
W20260806 01:14:30.713515 4120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:30.713824 4120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:30.713917 4120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:30.721871 4120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:30.722020 4120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:30.722077 4120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:30.722146 4120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:30.722196 4120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:14:30.732905 4120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:35167,127.25.236.189:34529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.189.keytab
--principal=oryx/127.25.236.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.189:34529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.236.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
W20260806 01:14:30.733305 4120 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.25.236.190:35167,127.25.236.189:34529'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 01:14:30.734882 4120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:30.737185 4120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:30.749965 4132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:30.750006 4129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:30.749980 4130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:30.755769 4120 server_base.cc:1034] running on GCE node
I20260806 01:14:30.756578 4120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:30.758126 4120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:30.759377 4120 hybrid_clock.cc:648] HybridClock initialized: now 1785978870759294 us; error 55 us; skew 500 ppm
I20260806 01:14:30.759745 4120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:30 dist-test-slave-7nm7 krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978870, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:30.763154 4120 init.cc:377] Logged in from keytab as oryx/127.25.236.189@KRBTEST.COM (short username oryx)
I20260806 01:14:30.764881 4120 webserver.cc:460] Webserver started at http://127.25.236.189:44567/ using document root <none> and password file <none>
I20260806 01:14:30.766206 4120 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:30.766371 4120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:30.766839 4120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:30.770761 4120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/data/instance:
uuid: "a703de63fe69433e876f4ef33496af84"
format_stamp: "Formatted at 2026-08-06 01:14:30 on dist-test-slave-7nm7"
server_key: "cd06b30778448894418b74d86c941ffe"
server_key_iv: "49607b4061d0a1e6011bc495d12fe282"
server_key_version: "encryptionkey@0"
I20260806 01:14:30.771595 4120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/wal/instance:
uuid: "a703de63fe69433e876f4ef33496af84"
format_stamp: "Formatted at 2026-08-06 01:14:30 on dist-test-slave-7nm7"
server_key: "cd06b30778448894418b74d86c941ffe"
server_key_iv: "49607b4061d0a1e6011bc495d12fe282"
server_key_version: "encryptionkey@0"
I20260806 01:14:30.777518 4120 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 01:14:30.781957 4148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:30.783977 4120 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260806 01:14:30.784299 4120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/wal
uuid: "a703de63fe69433e876f4ef33496af84"
format_stamp: "Formatted at 2026-08-06 01:14:30 on dist-test-slave-7nm7"
server_key: "cd06b30778448894418b74d86c941ffe"
server_key_iv: "49607b4061d0a1e6011bc495d12fe282"
server_key_version: "encryptionkey@0"
I20260806 01:14:30.784564 4120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:30.823833 4120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:30.828848 4120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:30.829159 4120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:30.851367 4120 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.189:34529
I20260806 01:14:30.851397 4230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.189:34529 every 8 connection(s)
I20260806 01:14:30.852794 4120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/data/info.pb
I20260806 01:14:30.856760 4231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:30.859465 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 4120
I20260806 01:14:30.866252 4231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:14:30 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.236.189@KRBTEST.COM
Aug 06 01:14:30 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978870, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.189@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:30.892428 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:30.862658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34330 (local address 127.25.236.189:34529)
0806 01:14:30.863670 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:30.863686 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:30.863739 (+ 53us) server_negotiation.cc:407] Connection header received
0806 01:14:30.866090 (+ 2351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:30.866128 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:30.866717 (+ 589us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:30.867179 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:30.869232 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.871439 (+ 2207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.873622 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.874890 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.875791 (+ 901us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:30.881127 (+ 5336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:30.881141 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:30.881161 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:30.881237 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:30.886057 (+ 4820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:30.887125 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:30.887136 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:30.887147 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:30.887399 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:30.887911 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:30.887916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:30.887920 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:30.888822 (+ 902us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:30.890187 (+ 1365us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:30.890514 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 01:14:30.890952 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":483,"threads_started":1}
I20260806 01:14:30.897485 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:30.869465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:47012)
0806 01:14:30.879791 (+ 10326us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:30.881162 (+ 1371us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:30.881349 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:30.881768 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:30.881775 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:30.882713 (+ 938us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:30.883011 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:30.883054 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.884506 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.884511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:30.885718 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:30.885730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.886518 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.886525 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:30.886714 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:30.887312 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:30.887330 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:30.890372 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:30.894004 (+ 3632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:30.894016 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:30.894023 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:30.894824 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:30.895436 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:30.895441 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:30.895444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:30.895610 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:30.896366 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:30.896373 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:30.896565 (+ 192us) client_negotiation.cc:769] Sending connection context
0806 01:14:30.896759 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 01:14:30.897071 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9993,"thread_start_us":9834,"threads_started":1}
I20260806 01:14:30.898392 4237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:30.869349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47012 (local address 127.25.236.190:35167)
0806 01:14:30.881231 (+ 11882us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:30.881237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:30.881258 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:14:30.881516 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:30.881521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:30.881610 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:30.881726 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:30.883206 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.884290 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.885863 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.886335 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.886475 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:30.890602 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:30.890608 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:30.890612 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:30.890653 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:30.893866 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:30.894968 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:30.894973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:30.894977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:30.895273 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:30.895741 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:30.895746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:30.895750 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:30.896153 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:30.896292 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:30.897783 (+ 1491us) server_negotiation.cc:299] Negotiation successful
0806 01:14:30.898000 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11748,"thread_start_us":11602,"threads_started":1}
I20260806 01:14:30.901759 4231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:14:30 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978870, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.189@KRBTEST.COM for oryx/127.25.236.189@KRBTEST.COM
I20260806 01:14:30.914220 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:30.902486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:34529 (local address 127.0.0.1:34346)
0806 01:14:30.902692 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:30.902710 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:30.902853 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:30.903126 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:30.903129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:30.903519 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:30.903646 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:30.903654 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.904654 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.904657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:30.905532 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:30.905542 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.906256 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.906263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:30.906395 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:30.906896 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:30.906912 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:30.908969 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:30.912037 (+ 3068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:30.912041 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:30.912044 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:30.912465 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:30.912844 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:30.912847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:30.912849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:30.912968 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:30.913528 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:30.913532 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:30.913647 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 01:14:30.913762 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 01:14:30.913972 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 01:14:30.914484 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:30.902512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34346 (local address 127.25.236.189:34529)
0806 01:14:30.902663 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:30.902667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:30.902784 (+ 117us) server_negotiation.cc:407] Connection header received
0806 01:14:30.902950 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:30.902954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:30.903007 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:30.903090 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:30.903784 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.904486 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.905662 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.906063 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.906174 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:30.909204 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:30.909209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:30.909213 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:30.909241 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:30.911898 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:30.912612 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:30.912616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:30.912619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:30.912738 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:30.913059 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:30.913062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:30.913064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:30.913396 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:30.913505 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:30.913852 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 01:14:30.914096 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":56}
I20260806 01:14:30.919735 4231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: Bootstrap starting.
I20260806 01:14:30.922832 4231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:30.923871 4231 log.cc:824] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:30.926550 4231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: No bootstrap required, opened a new log
I20260806 01:14:30.933653 4231 raft_consensus.cc:348] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } }
I20260806 01:14:30.933951 4231 raft_consensus.cc:374] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:30.934075 4231 raft_consensus.cc:729] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a703de63fe69433e876f4ef33496af84, State: Initialized, Role: FOLLOWER
I20260806 01:14:30.934732 4231 consensus_queue.cc:260] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } }
I20260806 01:14:30.935637 4240 sys_catalog.cc:455] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } } }
I20260806 01:14:30.935995 4240 sys_catalog.cc:458] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:14:30.936656 4231 sys_catalog.cc:564] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 01:14:30.947059 4260 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:14:30.947237 4260 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:14:30.951340 4120 master_runner.cc:428] Detected that this master a703de63fe69433e876f4ef33496af84 is joining an existing cluster
I20260806 01:14:30.951436 4120 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.236.189:34529
I20260806 01:14:30.987344 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:30.957216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34356 (local address 127.25.236.189:34529)
0806 01:14:30.957455 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:30.957463 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:30.967008 (+ 9545us) server_negotiation.cc:407] Connection header received
0806 01:14:30.967225 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:30.967233 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:30.967337 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:30.967478 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:30.969479 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.971060 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.974257 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.975361 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.975506 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:30.978973 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:30.978980 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:30.978986 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:30.979025 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:30.983680 (+ 4655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:30.984708 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:30.984717 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:30.984725 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:30.984909 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:30.985390 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:30.985395 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:30.985397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:30.985785 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:30.985933 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:30.986558 (+ 625us) server_negotiation.cc:299] Negotiation successful
0806 01:14:30.986846 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:14:30.988879 4270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:30.957309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:34529 (local address 127.0.0.1:34356)
0806 01:14:30.966820 (+ 9511us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:30.966852 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:30.967071 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:30.967712 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:30.967720 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:30.968732 (+ 1012us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:30.969258 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:30.969278 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.971368 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.971376 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:30.973994 (+ 2618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:30.974021 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.975971 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.975980 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:30.976246 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:30.977110 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:30.977143 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:30.978596 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:30.983911 (+ 5315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:30.983918 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:30.983925 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:30.984548 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:30.985062 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:30.985067 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:30.985071 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:30.985257 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:30.986004 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:30.986011 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:30.986189 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 01:14:30.987865 (+ 1676us) client_negotiation.cc:240] Negotiation successful
0806 01:14:30.988289 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9288,"thread_start_us":9106,"threads_started":1}
WARNING: no policy specified for oryx/127.25.236.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.236.188@KRBTEST.COM" created.
I20260806 01:14:30.996908 4237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:30.957231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47028 (local address 127.25.236.190:35167)
0806 01:14:30.957440 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:30.957449 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:30.978038 (+ 20589us) server_negotiation.cc:407] Connection header received
0806 01:14:30.978736 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:30.978745 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:30.978908 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:30.978994 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:30.982410 (+ 3416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.983644 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.985826 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.986564 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.986696 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:30.990024 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:30.990031 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:30.990036 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:30.990098 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:30.993753 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:30.994822 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:30.994828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:30.994832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:30.994985 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:30.995395 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:30.995399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:30.995401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:30.995693 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:30.995859 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:30.996305 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 01:14:30.996547 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:14:30.997663 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:30.957826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:47028)
0806 01:14:30.977905 (+ 20079us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:30.977930 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:30.978510 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:30.979605 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:30.979613 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:30.981371 (+ 1758us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:30.982138 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:30.982158 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.983903 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.983910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:30.985667 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:30.985680 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:30.986968 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:30.986976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:30.987186 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:30.987813 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:30.987836 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:30.989271 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:30.993914 (+ 4643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:30.993919 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:30.993921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:30.994627 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:30.995111 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:30.995118 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:30.995123 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:30.995259 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:30.995820 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:30.995824 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:30.996038 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 01:14:30.996181 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 01:14:30.996525 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19841,"thread_start_us":19408,"threads_started":1}
Aug 06 01:14:31 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978868, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.190@KRBTEST.COM for oryx/127.25.236.189@KRBTEST.COM
I20260806 01:14:31.028434 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.006574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.25.236.189:34529)
0806 01:14:31.006839 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:31.006848 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:31.009940 (+ 3092us) server_negotiation.cc:407] Connection header received
0806 01:14:31.010376 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:31.010380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:31.010459 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:31.010584 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:31.011807 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.012535 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.014452 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.015207 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.015381 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.020569 (+ 5188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:31.020575 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:31.020579 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:31.020628 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:31.025064 (+ 4436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.026089 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.026094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.026096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.026312 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.026762 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.026767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.026770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.027100 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:31.027280 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:31.027782 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 01:14:31.028042 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 01:14:31.028487 4274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.006463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:34529 (local address 127.0.0.1:34358)
0806 01:14:31.009783 (+ 3320us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:31.009835 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:31.010099 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:31.010677 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:31.010685 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:31.011378 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:31.011607 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.011627 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.012681 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.012687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.014074 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.014107 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.015474 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.015489 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.015944 (+ 455us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.017070 (+ 1126us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:31.017089 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:31.020303 (+ 3214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:31.025287 (+ 4984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.025303 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.025312 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.025935 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.026430 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.026435 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.026439 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.026634 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.027253 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:31.027262 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:31.027463 (+ 201us) client_negotiation.cc:769] Sending connection context
0806 01:14:31.027684 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 01:14:31.028033 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3047,"thread_start_us":2893,"threads_started":1}
I20260806 01:14:31.030840 3384 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.236.189:34529: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a703de63fe69433e876f4ef33496af84"
member_type: NON_VOTER
last_known_addr {
host: "127.25.236.189"
port: 34529
}
attrs {
promote: true
}
}
dest_uuid: "d0dac0217f664480abd32c52e2e36380"
cas_config_opid_index: -1
I20260806 01:14:31.032122 3384 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true } }
I20260806 01:14:31.036985 4195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:31.036629 4280 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a703de63fe69433e876f4ef33496af84 (127.25.236.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true } } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.236.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.188.keytab.
Entry for principal oryx/127.25.236.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.188.keytab.
I20260806 01:14:31.040421 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.188:40513
--webserver_interface=127.25.236.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.188.keytab
--principal=oryx/127.25.236.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:35167,127.25.236.189:34529,127.25.236.188:40513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:14:31.041733 4195 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 FOLLOWER]: Refusing update from remote peer d0dac0217f664480abd32c52e2e36380: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 01:14:31.041802 4280 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.236.189:34529
I20260806 01:14:31.042902 4120 master.cc:537] Master@127.25.236.189:34529 shutting down...
I20260806 01:14:31.042747 4280 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true } } }
I20260806 01:14:31.042990 4280 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:31.047060 4280 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:14:31.050634 3361 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 01:14:31.052603 3361 consensus_peers.cc:487] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 -> Peer a703de63fe69433e876f4ef33496af84 (127.25.236.189:34529): Couldn't send request to peer a703de63fe69433e876f4ef33496af84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 01:14:31.059257 4120 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:31.059753 4120 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:31.060349 4120 tablet_replica.cc:330] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: stopping tablet replica
I20260806 01:14:31.068789 4120 master.cc:552] Master@127.25.236.189:34529 shutdown complete.
I20260806 01:14:31.068934 4120 master_runner.cc:305] Clearing existing system tablet
I20260806 01:14:31.069479 4120 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:31.069582 4120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:31.072686 4120 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:14:31.074930 4288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:31.075582 4120 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:31.075877 4120 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 01:14:31.075968 4120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/wal
uuid: "a703de63fe69433e876f4ef33496af84"
format_stamp: "Formatted at 2026-08-06 01:14:30 on dist-test-slave-7nm7"
server_key: "cd06b30778448894418b74d86c941ffe"
server_key_iv: "49607b4061d0a1e6011bc495d12fe282"
server_key_version: "encryptionkey@0"
I20260806 01:14:31.081115 4120 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:31.088752 4120 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 01:14:31.088963 4120 log.cc:1196] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 01:14:31.089450 4120 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: Deleting consensus metadata
I20260806 01:14:31.089756 4120 master_runner.cc:315] Copying system tablet from 127.25.236.190:35167
I20260806 01:14:31.093772 4120 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: tablet copy: Beginning tablet copy session from remote peer at address 127.25.236.190:35167
I20260806 01:14:31.114620 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.094788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:47030)
0806 01:14:31.098704 (+ 3916us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:31.098726 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:31.098927 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:31.099346 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:31.099353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:31.099930 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:31.100225 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.100238 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.101661 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.101666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.103276 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.103291 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.104762 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.104769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.104964 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.105744 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:31.105770 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:31.106907 (+ 1137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:31.111329 (+ 4422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.111336 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.111340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.111884 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.112375 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.112380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.112384 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.112558 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.113277 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:31.113284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:31.113478 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 01:14:31.113657 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 01:14:31.113984 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3769,"thread_start_us":3649,"threads_started":1}
I20260806 01:14:31.115638 4237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.094835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47030 (local address 127.25.236.190:35167)
0806 01:14:31.095017 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:31.095023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:31.098839 (+ 3816us) server_negotiation.cc:407] Connection header received
0806 01:14:31.099093 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:31.099099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:31.099183 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:31.099302 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:31.100398 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.101490 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.103452 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.104288 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.104406 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.107172 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:31.107181 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:31.107186 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:31.107232 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:31.111146 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.112048 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.112055 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.112061 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.112244 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.112707 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.112715 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.112720 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.113136 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:31.113313 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:31.115008 (+ 1695us) server_negotiation.cc:299] Negotiation successful
0806 01:14:31.115254 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:14:31.116911 3418 tablet_copy_service.cc:140] P d0dac0217f664480abd32c52e2e36380: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a703de63fe69433e876f4ef33496af84 ({username='oryx', principal='oryx/127.25.236.189@KRBTEST.COM'} at 127.0.0.1:47030)
I20260806 01:14:31.117545 3418 tablet_copy_service.cc:161] P d0dac0217f664480abd32c52e2e36380: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a703de63fe69433e876f4ef33496af84 at {username='oryx', principal='oryx/127.25.236.189@KRBTEST.COM'} at 127.0.0.1:47030: session id = a703de63fe69433e876f4ef33496af84-00000000000000000000000000000000
I20260806 01:14:31.124622 3418 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 01:14:31.128134 4120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 01:14:31.135805 4120 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: tablet copy: Starting download of 0 data blocks...
I20260806 01:14:31.136260 4120 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: tablet copy: Starting download of 1 WAL segments...
I20260806 01:14:31.141024 4120 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 01:14:31.147349 3418 tablet_copy_service.cc:342] P d0dac0217f664480abd32c52e2e36380: Request end of tablet copy session a703de63fe69433e876f4ef33496af84-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.236.189@KRBTEST.COM'} at 127.0.0.1:47030
I20260806 01:14:31.147753 3418 tablet_copy_service.cc:433] P d0dac0217f664480abd32c52e2e36380: ending tablet copy session a703de63fe69433e876f4ef33496af84-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a703de63fe69433e876f4ef33496af84
I20260806 01:14:31.160933 4120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:31.163648 4297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:31.169327 4298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:31.176956 4302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:31.177734 4120 server_base.cc:1034] running on GCE node
I20260806 01:14:31.178196 4120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:31.179109 4120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:31.180378 4120 hybrid_clock.cc:648] HybridClock initialized: now 1785978871180308 us; error 37 us; skew 500 ppm
I20260806 01:14:31.180647 4120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:31 dist-test-slave-7nm7 krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978871, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:31.183807 4120 init.cc:377] Logged in from keytab as oryx/127.25.236.189@KRBTEST.COM (short username oryx)
I20260806 01:14:31.187198 4120 webserver.cc:460] Webserver started at http://127.25.236.189:36493/ using document root <none> and password file <none>
I20260806 01:14:31.187552 4120 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:31.187647 4120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:31.190203 4120 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 01:14:31.191988 4309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:31.192422 4120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 01:14:31.192575 4120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/wal
uuid: "a703de63fe69433e876f4ef33496af84"
format_stamp: "Formatted at 2026-08-06 01:14:30 on dist-test-slave-7nm7"
server_key: "cd06b30778448894418b74d86c941ffe"
server_key_iv: "49607b4061d0a1e6011bc495d12fe282"
server_key_version: "encryptionkey@0"
I20260806 01:14:31.192749 4120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:31.212555 4120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:31.217532 4120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:31.261165 4120 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.189:34529
I20260806 01:14:31.261189 4381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.189:34529 every 8 connection(s)
I20260806 01:14:31.262153 4120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-1/data/info.pb
I20260806 01:14:31.266422 4112 heartbeater.cc:503] Master 127.25.236.190:35167 was elected leader, sending a full tablet report...
I20260806 01:14:31.268702 4384 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:14:31.272857 4384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: Bootstrap starting.
I20260806 01:14:31.289126 4384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:14:31.289683 4384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: Bootstrap complete.
I20260806 01:14:31.290668 4384 raft_consensus.cc:348] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true } }
I20260806 01:14:31.290920 4384 raft_consensus.cc:729] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a703de63fe69433e876f4ef33496af84, State: Initialized, Role: LEARNER
I20260806 01:14:31.291174 4384 consensus_queue.cc:260] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true } }
I20260806 01:14:31.292030 4387 sys_catalog.cc:455] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true } } }
I20260806 01:14:31.292356 4384 sys_catalog.cc:564] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:31.292506 4387 sys_catalog.cc:458] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:14:31.306020 4400 catalog_manager.cc:1259] Loaded cluster ID: 48e3156596764ed7a8f5e71c7c835421
I20260806 01:14:31.306170 4400 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: loading cluster ID for follower catalog manager: success
I20260806 01:14:31.308751 4400 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: acquiring CA information for follower catalog manager: success
I20260806 01:14:31.310940 4400 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 01:14:31.523470 4285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:31.523834 4285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:31.524024 4285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:31.532621 4285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:31.532789 4285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:31.532895 4285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:31.533053 4285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:31.533139 4285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:14:31.546088 4285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:35167,127.25.236.189:34529,127.25.236.188:40513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.188.keytab
--principal=oryx/127.25.236.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.188:40513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.236.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:31.547676 4285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:31.549295 4285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:31.563525 4406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:31.563557 4413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:31.563618 4409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:31.564689 4285 server_base.cc:1034] running on GCE node
I20260806 01:14:31.565497 4285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:31.567024 4285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:31.568269 4285 hybrid_clock.cc:648] HybridClock initialized: now 1785978871568219 us; error 42 us; skew 500 ppm
I20260806 01:14:31.568668 4285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:31 dist-test-slave-7nm7 krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978871, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:31.571987 4285 init.cc:377] Logged in from keytab as oryx/127.25.236.188@KRBTEST.COM (short username oryx)
I20260806 01:14:31.573527 4285 webserver.cc:460] Webserver started at http://127.25.236.188:32775/ using document root <none> and password file <none>
I20260806 01:14:31.574148 4285 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:31.574317 4285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:31.574652 4285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:31.577487 4285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/data/instance:
uuid: "d446404a648949cea8e2c4eef9464be6"
format_stamp: "Formatted at 2026-08-06 01:14:31 on dist-test-slave-7nm7"
server_key: "7b97290a704db907e5c550ee7d8fb1d4"
server_key_iv: "ddc3d58637e093772759d24ec3c40a51"
server_key_version: "encryptionkey@0"
I20260806 01:14:31.578220 4285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/wal/instance:
uuid: "d446404a648949cea8e2c4eef9464be6"
format_stamp: "Formatted at 2026-08-06 01:14:31 on dist-test-slave-7nm7"
server_key: "7b97290a704db907e5c550ee7d8fb1d4"
server_key_iv: "ddc3d58637e093772759d24ec3c40a51"
server_key_version: "encryptionkey@0"
I20260806 01:14:31.584372 4285 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 01:14:31.588191 4425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:31.589830 4285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:14:31.589973 4285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/wal
uuid: "d446404a648949cea8e2c4eef9464be6"
format_stamp: "Formatted at 2026-08-06 01:14:31 on dist-test-slave-7nm7"
server_key: "7b97290a704db907e5c550ee7d8fb1d4"
server_key_iv: "ddc3d58637e093772759d24ec3c40a51"
server_key_version: "encryptionkey@0"
I20260806 01:14:31.590142 4285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:31.601162 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.594177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:34529 (local address 127.0.0.1:34362)
0806 01:14:31.594687 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:31.594709 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:31.594851 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:31.595776 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:31.595782 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:31.595818 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:31.596000 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.596009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.597692 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.597698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.599053 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.599067 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.600309 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.600317 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.600483 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.600560 (+ 77us) client_negotiation.cc:769] Sending connection context
0806 01:14:31.600667 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 01:14:31.600815 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":456,"threads_started":1}
I20260806 01:14:31.601182 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.594425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34362 (local address 127.25.236.189:34529)
0806 01:14:31.595313 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:31.595318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:31.595405 (+ 87us) server_negotiation.cc:407] Connection header received
0806 01:14:31.595498 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:31.595502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:31.595577 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:31.595726 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:31.596172 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.597533 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.599225 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.600121 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.600332 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.600502 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:31.600705 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 01:14:31.600836 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":785,"thread_start_us":646,"threads_started":1}
I20260806 01:14:31.602388 4349 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260806 01:14:31.602973 4387 sys_catalog.cc:455] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0dac0217f664480abd32c52e2e36380. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true } } }
I20260806 01:14:31.603266 4387 sys_catalog.cc:458] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:14:31.620111 4285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:31.625334 4285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:31.625674 4285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:31.649448 4285 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.188:40513
I20260806 01:14:31.649540 4515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.188:40513 every 8 connection(s)
I20260806 01:14:31.650955 4285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/data/info.pb
I20260806 01:14:31.655175 4516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:31.656409 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 4285
I20260806 01:14:31.666829 4516 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:14:31 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.236.188@KRBTEST.COM
Aug 06 01:14:31 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978871, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.188@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:31.688639 4522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.658955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45220 (local address 127.25.236.188:40513)
0806 01:14:31.660239 (+ 1284us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:31.660250 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:31.660309 (+ 59us) server_negotiation.cc:407] Connection header received
0806 01:14:31.662746 (+ 2437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:31.662768 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:31.663154 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:31.663506 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:31.665811 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.667493 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.669619 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.671112 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.671318 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.677869 (+ 6551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:31.677878 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:31.677888 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:31.677922 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:31.682716 (+ 4794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.683642 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.683659 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.683675 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.684078 (+ 403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.684871 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.684878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.684882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.685724 (+ 842us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:31.686141 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:31.686670 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 01:14:31.687133 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1045,"thread_start_us":668,"threads_started":1}
I20260806 01:14:31.691169 4524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.670563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:47036)
0806 01:14:31.671589 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:31.671669 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:31.671871 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:31.672354 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:31.672359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:31.673446 (+ 1087us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:31.673903 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.673926 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.675086 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.675091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.676912 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.676939 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.677891 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.677899 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.678063 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.678921 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:31.678939 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:31.682771 (+ 3832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:31.687769 (+ 4998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.687781 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.687787 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.688366 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.688969 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.688974 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.688977 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.689146 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.689835 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:31.689842 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:31.690038 (+ 196us) client_negotiation.cc:769] Sending connection context
0806 01:14:31.690338 (+ 300us) client_negotiation.cc:240] Negotiation successful
0806 01:14:31.690669 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":484,"threads_started":1}
I20260806 01:14:31.691195 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.670134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47036 (local address 127.25.236.190:35167)
0806 01:14:31.670830 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:31.670834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:31.671776 (+ 942us) server_negotiation.cc:407] Connection header received
0806 01:14:31.672021 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:31.672025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:31.672135 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:31.672253 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:31.674084 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.674925 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.677108 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.677510 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.677620 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.683006 (+ 5386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:31.683012 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:31.683015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:31.683054 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:31.687601 (+ 4547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.688504 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.688511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.688514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.688745 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.689286 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.689291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.689293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.689694 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:31.689831 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:31.690444 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 01:14:31.690664 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":462,"threads_started":1}
I20260806 01:14:31.692943 4516 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:14:31 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978871, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.188@KRBTEST.COM for oryx/127.25.236.189@KRBTEST.COM
I20260806 01:14:31.709612 4524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.694032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:34529 (local address 127.0.0.1:34366)
0806 01:14:31.694384 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:31.694410 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:31.694545 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:31.695035 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:31.695038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:31.695491 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:31.695835 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.695844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.697459 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.697464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.698923 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.698936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.699863 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.699868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.700073 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.700861 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:31.700879 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:31.703381 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:31.706988 (+ 3607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.706992 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.706994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.707516 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.707913 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.707917 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.707919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.708158 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.708779 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:31.708783 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:31.708942 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 01:14:31.709053 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 01:14:31.709341 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 01:14:31.710304 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.694054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34366 (local address 127.25.236.189:34529)
0806 01:14:31.694379 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:31.694387 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:31.694472 (+ 85us) server_negotiation.cc:407] Connection header received
0806 01:14:31.694751 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:31.694759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:31.694852 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:31.694964 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:31.696045 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.697275 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.699080 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.699672 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.699773 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.703580 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:31.703585 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:31.703589 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:31.703620 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:31.706850 (+ 3230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.707644 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.707648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.707650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.707775 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.708298 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.708302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.708304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.708642 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:31.708801 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:31.709237 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 01:14:31.709571 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 01:14:31.711434 4516 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:14:31 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978871, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.188@KRBTEST.COM for oryx/127.25.236.188@KRBTEST.COM
I20260806 01:14:31.727696 4522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.712397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.25.236.188:40513)
0806 01:14:31.712599 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:31.712603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:31.712617 (+ 14us) server_negotiation.cc:407] Connection header received
0806 01:14:31.712655 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:31.712658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:31.712708 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:31.712784 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:31.713601 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.714431 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.716193 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.716805 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.716931 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.721718 (+ 4787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:31.721723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:31.721726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:31.721756 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:31.725022 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.725698 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.725704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.725708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.725818 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.726221 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.726253 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.726259 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.726552 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:31.726716 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:31.727069 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 01:14:31.727277 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:14:31.727969 4524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.712184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:40513 (local address 127.0.0.1:45232)
0806 01:14:31.712377 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:31.712392 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:31.712487 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:31.712830 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:31.712834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:31.713286 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:31.713443 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.713451 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.714595 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.714600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.715862 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.715904 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.717003 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.717008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.717445 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.718621 (+ 1176us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:31.718644 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:31.721502 (+ 2858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:31.725184 (+ 3682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.725190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.725196 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.725573 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.725922 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.725928 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.725931 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.726086 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.726678 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:31.726683 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:31.726844 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 01:14:31.726986 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 01:14:31.727408 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":143}
I20260806 01:14:31.733873 4516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: Bootstrap starting.
I20260806 01:14:31.738004 4516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:31.739660 4516 log.cc:824] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:31.743984 4516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: No bootstrap required, opened a new log
I20260806 01:14:31.752424 4516 raft_consensus.cc:348] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } }
I20260806 01:14:31.752910 4516 raft_consensus.cc:374] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:31.753074 4516 raft_consensus.cc:729] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d446404a648949cea8e2c4eef9464be6, State: Initialized, Role: FOLLOWER
I20260806 01:14:31.753814 4516 consensus_queue.cc:260] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } }
I20260806 01:14:31.754946 4527 sys_catalog.cc:455] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } } }
I20260806 01:14:31.755350 4527 sys_catalog.cc:458] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:14:31.756245 4516 sys_catalog.cc:564] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 01:14:31.769847 4540 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:14:31.770076 4540 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:14:31.774806 4285 master_runner.cc:428] Detected that this master d446404a648949cea8e2c4eef9464be6 is joining an existing cluster
I20260806 01:14:31.775044 4285 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.236.188:40513
I20260806 01:14:31.795413 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 3481
I20260806 01:14:31.804225 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.780473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34380 (local address 127.25.236.189:34529)
0806 01:14:31.780682 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:31.780691 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:31.781631 (+ 940us) server_negotiation.cc:407] Connection header received
0806 01:14:31.781956 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:31.781964 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:31.782060 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:31.782162 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:31.784758 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.786344 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.789091 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.789740 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.789846 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.794114 (+ 4268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:31.794133 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:31.794139 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:31.794186 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:31.799079 (+ 4893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.800675 (+ 1596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.800682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.800686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.800952 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.801619 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.801625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.801628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.802092 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:31.802262 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:31.803400 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0806 01:14:31.803777 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:14:31.805913 4551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.780475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:34529 (local address 127.0.0.1:34380)
0806 01:14:31.781505 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:31.781531 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:31.781758 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:31.782687 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:31.782692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:31.783972 (+ 1280us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:31.784518 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.784535 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.786603 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.786612 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.788816 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.788856 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.790850 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.790862 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.791121 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.792063 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:31.792091 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:31.793873 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:31.799516 (+ 5643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.799522 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.799526 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.800493 (+ 967us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.801156 (+ 663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.801163 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.801170 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.801468 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.802383 (+ 915us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:31.802389 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:31.802668 (+ 279us) client_negotiation.cc:769] Sending connection context
0806 01:14:31.804730 (+ 2062us) client_negotiation.cc:240] Negotiation successful
0806 01:14:31.805320 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"thread_start_us":415,"threads_started":1}
I20260806 01:14:31.817512 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.780309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47040 (local address 127.25.236.190:35167)
0806 01:14:31.780527 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:31.780534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:31.782859 (+ 2325us) server_negotiation.cc:407] Connection header received
0806 01:14:31.786655 (+ 3796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:31.786663 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:31.786798 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:31.786881 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:31.791023 (+ 4142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.792474 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.794749 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.795492 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.795612 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.807678 (+ 12066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:31.807689 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:31.807697 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:31.807760 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:31.812728 (+ 4968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.813740 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.813748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.813751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.813922 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.814486 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.814492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.814495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.815297 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:31.815484 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:31.816282 (+ 798us) server_negotiation.cc:299] Negotiation successful
0806 01:14:31.816853 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 01:14:31.819201 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.781387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:47040)
0806 01:14:31.782693 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:31.782711 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:31.786449 (+ 3738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:31.787136 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:31.787141 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:31.788588 (+ 1447us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:31.790765 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.790780 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.792850 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.792858 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.794569 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.794590 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.795972 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.795981 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.796521 (+ 540us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.799356 (+ 2835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:31.799385 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:31.807369 (+ 7984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:31.812912 (+ 5543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.812918 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.812922 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.813576 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.814083 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.814088 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.814091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.814335 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.815580 (+ 1245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:31.815585 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:31.815882 (+ 297us) client_negotiation.cc:769] Sending connection context
0806 01:14:31.817881 (+ 1999us) client_negotiation.cc:240] Negotiation successful
0806 01:14:31.818511 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"thread_start_us":600,"threads_started":1}
Aug 06 01:14:31 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.236.129@KRBTEST.COM
I20260806 01:14:31.823380 4553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.782576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:40513 (local address 127.0.0.1:45246)
0806 01:14:31.790978 (+ 8402us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:31.790998 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:31.791183 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:31.791656 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:31.791663 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:31.792699 (+ 1036us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:31.798759 (+ 6060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.798780 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.803804 (+ 5024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.803813 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.806604 (+ 2791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.806629 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.810066 (+ 3437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.810071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.810324 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.812278 (+ 1954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:31.812309 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:31.813592 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:31.818577 (+ 4985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.818581 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.818584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.819246 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.819736 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.819742 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.819748 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.819899 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.820671 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:31.820677 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:31.820902 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 01:14:31.821096 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 01:14:31.821529 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8137,"thread_start_us":6435,"threads_started":1}
I20260806 01:14:31.823391 4522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.782663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45246 (local address 127.25.236.188:40513)
0806 01:14:31.782910 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:31.782919 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:31.791065 (+ 8146us) server_negotiation.cc:407] Connection header received
0806 01:14:31.791320 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:31.791327 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:31.791390 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:31.791454 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:31.799957 (+ 8503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.801933 (+ 1976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.808773 (+ 6840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.809663 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.809759 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.813769 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:31.813775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:31.813780 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:31.813810 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:31.818432 (+ 4622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.819393 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.819398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.819402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.819578 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.820053 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.820058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.820062 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.820508 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:31.820672 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:31.821583 (+ 911us) server_negotiation.cc:299] Negotiation successful
0806 01:14:31.821827 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 01:14:31.835664 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.798988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60230 (local address 127.25.236.129:44437)
0806 01:14:31.809749 (+ 10761us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:31.809759 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:31.809790 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:14:31.809867 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:31.809877 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:31.810194 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:31.810780 (+ 586us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:31.812143 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.814038 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.816936 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.818052 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.822578 (+ 4526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.824469 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:31.824486 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:31.824507 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:31.824560 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:31.830539 (+ 5979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.831792 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.831813 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.831817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.832000 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.832594 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.832599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.832603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.833049 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:31.833389 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:31.834616 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0806 01:14:31.834966 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10518,"thread_start_us":11340,"threads_started":1}
I20260806 01:14:31.838089 3687 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 06 01:14:31 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978868, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.190@KRBTEST.COM for oryx/127.25.236.188@KRBTEST.COM
I20260806 01:14:31.851749 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.828523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:40513 (local address 127.0.0.1:45250)
0806 01:14:31.828734 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:31.828756 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:31.828869 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:31.829555 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:31.829561 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:31.830062 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:31.830357 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.830372 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.831610 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.831625 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.833195 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.833209 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.834862 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.834872 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.835124 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.835971 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:31.835997 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:31.840264 (+ 4267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:31.848471 (+ 8207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.848478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.848483 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.849118 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.849624 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.849632 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.849637 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.849837 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.850564 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:31.850569 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:31.850751 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 01:14:31.850889 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 01:14:31.851224 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 01:14:31.853222 4522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.828967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45250 (local address 127.25.236.188:40513)
0806 01:14:31.829164 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:31.829171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:31.829193 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:14:31.829301 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:31.829306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:31.829385 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:31.829501 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:31.830529 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.831473 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.833370 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.834580 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.840612 (+ 6032us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.841036 (+ 424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:31.841043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:31.841050 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:31.841175 (+ 125us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:31.848237 (+ 7062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.849315 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.849320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.849324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.849487 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.850000 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.850005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.850009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.850413 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:31.852410 (+ 1997us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:31.852590 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 01:14:31.852844 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:14:31.855439 3384 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.236.188:40513: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d446404a648949cea8e2c4eef9464be6"
member_type: NON_VOTER
last_known_addr {
host: "127.25.236.188"
port: 40513
}
attrs {
promote: true
}
}
dest_uuid: "d0dac0217f664480abd32c52e2e36380"
cas_config_opid_index: 5
I20260806 01:14:31.856870 3384 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } }
I20260806 01:14:31.860148 4430 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d446404a648949cea8e2c4eef9464be6 (127.25.236.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } } }
I20260806 01:14:31.863358 4349 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 LEARNER]: Refusing update from remote peer d0dac0217f664480abd32c52e2e36380: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 01:14:31.865729 4476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:31.868826 4557 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 01:14:31.870648 4476 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 1 FOLLOWER]: Refusing update from remote peer d0dac0217f664480abd32c52e2e36380: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 01:14:31.871495 4430 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.236.188:40513
I20260806 01:14:31.879467 4563 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 01:14:31.880550 4349 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d446404a648949cea8e2c4eef9464be6 (127.25.236.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } } }
I20260806 01:14:31.881592 4557 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } } }
W20260806 01:14:31.882050 4446 tablet.cc:2369] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:14:31.882386 4285 master.cc:537] Master@127.25.236.188:40513 shutting down...
I20260806 01:14:31.882371 4557 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:31.888128 4387 sys_catalog.cc:455] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: NON_VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: true } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } } }
I20260806 01:14:31.888556 4387 sys_catalog.cc:458] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:14:31.890352 4430 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380: attempting to promote NON_VOTER a703de63fe69433e876f4ef33496af84 to VOTER
I20260806 01:14:31.891119 4430 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } }
I20260806 01:14:31.893532 4349 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 LEARNER]: Refusing update from remote peer d0dac0217f664480abd32c52e2e36380: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 01:14:31.894852 4563 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 01:14:31.899884 4558 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a703de63fe69433e876f4ef33496af84 (127.25.236.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } } }
I20260806 01:14:31.904026 4558 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } } }
I20260806 01:14:31.904361 4558 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:31.904479 4559 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } } }
I20260806 01:14:31.904851 4559 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:31.906013 4349 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a703de63fe69433e876f4ef33496af84 (127.25.236.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } } }
I20260806 01:14:31.912793 4387 sys_catalog.cc:455] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } } }
I20260806 01:14:31.913059 4387 sys_catalog.cc:458] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:14:31.919883 4285 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:31.920727 4285 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:31.921118 4285 tablet_replica.cc:330] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: stopping tablet replica
I20260806 01:14:31.929776 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.929305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:40513 (local address 127.0.0.1:45258)
0806 01:14:31.929486 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:31.929622 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:40513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 01:14:31.929765 4285 master.cc:552] Master@127.25.236.188:40513 shutdown complete.
I20260806 01:14:31.929883 4285 master_runner.cc:305] Clearing existing system tablet
W20260806 01:14:31.930022 3364 consensus_peers.cc:487] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 -> Peer d446404a648949cea8e2c4eef9464be6 (127.25.236.188:40513): Couldn't send request to peer d446404a648949cea8e2c4eef9464be6. Status: Network error: Client connection negotiation failed: client connection to 127.25.236.188:40513: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 01:14:31.930332 4285 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:31.930452 4285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:31.933763 4285 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 01:14:31.935519 4569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:31.935954 4285 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:31.936601 4285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 01:14:31.936729 4285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/wal
uuid: "d446404a648949cea8e2c4eef9464be6"
format_stamp: "Formatted at 2026-08-06 01:14:31 on dist-test-slave-7nm7"
server_key: "7b97290a704db907e5c550ee7d8fb1d4"
server_key_iv: "ddc3d58637e093772759d24ec3c40a51"
server_key_version: "encryptionkey@0"
I20260806 01:14:31.942448 4285 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:31.951188 4285 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 01:14:31.951390 4285 log.cc:1196] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 01:14:31.951942 4285 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: Deleting consensus metadata
I20260806 01:14:31.952302 4285 master_runner.cc:315] Copying system tablet from 127.25.236.190:35167
I20260806 01:14:31.956123 4285 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: tablet copy: Beginning tablet copy session from remote peer at address 127.25.236.190:35167
I20260806 01:14:31.976898 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.957687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47056 (local address 127.25.236.190:35167)
0806 01:14:31.957930 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:31.957937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:31.958868 (+ 931us) server_negotiation.cc:407] Connection header received
0806 01:14:31.959099 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:31.959106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:31.959199 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:31.959334 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:31.960815 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.961974 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.964038 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.964794 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.964933 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.968473 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:31.968480 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:31.968485 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:31.968526 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:31.972336 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.973691 (+ 1355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.973697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.973701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.973878 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:31.974775 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:31.974780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:31.974785 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:31.975354 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:31.975538 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:31.976105 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 01:14:31.976393 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260806 01:14:31.976933 4576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:31.957692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:47056)
0806 01:14:31.958716 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:31.958738 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:31.958929 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:31.959398 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:31.959409 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:31.960214 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:31.960643 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.960658 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.962169 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.962175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.963874 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:31.963889 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:31.965022 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:31.965029 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:31.965328 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:31.966286 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:31.966310 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:31.968215 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:31.972488 (+ 4273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.972494 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.972499 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.973384 (+ 885us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.974022 (+ 638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:31.974028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:31.974032 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:31.974605 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:31.975515 (+ 910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:31.975522 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:31.975754 (+ 232us) client_negotiation.cc:769] Sending connection context
0806 01:14:31.975963 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 01:14:31.976395 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"thread_start_us":609,"threads_started":1}
I20260806 01:14:31.978044 3418 tablet_copy_service.cc:140] P d0dac0217f664480abd32c52e2e36380: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d446404a648949cea8e2c4eef9464be6 ({username='oryx', principal='oryx/127.25.236.188@KRBTEST.COM'} at 127.0.0.1:47056)
I20260806 01:14:31.978271 3418 tablet_copy_service.cc:161] P d0dac0217f664480abd32c52e2e36380: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d446404a648949cea8e2c4eef9464be6 at {username='oryx', principal='oryx/127.25.236.188@KRBTEST.COM'} at 127.0.0.1:47056: session id = d446404a648949cea8e2c4eef9464be6-00000000000000000000000000000000
I20260806 01:14:31.982542 3418 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 01:14:31.984537 4285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 01:14:31.985090 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 3481
I20260806 01:14:31.992561 4285 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: tablet copy: Starting download of 0 data blocks...
I20260806 01:14:31.993119 4285 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: tablet copy: Starting download of 1 WAL segments...
I20260806 01:14:32.001530 4285 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 01:14:32.006951 3418 tablet_copy_service.cc:342] P d0dac0217f664480abd32c52e2e36380: Request end of tablet copy session d446404a648949cea8e2c4eef9464be6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.236.188@KRBTEST.COM'} at 127.0.0.1:47056
I20260806 01:14:32.007141 3418 tablet_copy_service.cc:433] P d0dac0217f664480abd32c52e2e36380: ending tablet copy session d446404a648949cea8e2c4eef9464be6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d446404a648949cea8e2c4eef9464be6
I20260806 01:14:32.018966 4285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:32.022040 4577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:32.024183 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:44437
--local_ip_for_outbound_sockets=127.25.236.129
--tserver_master_addrs=127.25.236.190:35167
--webserver_port=46879
--webserver_interface=127.25.236.129
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab
--principal=oryx/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.236.190:35167,127.25.236.189:34529
--tserver_master_addrs=127.25.236.190:35167,127.25.236.189:34529,127.25.236.188:40513 with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:32.028502 4578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:32.033679 4285 server_base.cc:1034] running on GCE node
W20260806 01:14:32.033969 4580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:32.034468 4285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:32.035252 4285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:32.036473 4285 hybrid_clock.cc:648] HybridClock initialized: now 1785978872036430 us; error 34 us; skew 500 ppm
I20260806 01:14:32.036777 4285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:32 dist-test-slave-7nm7 krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978872, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:32.039383 4285 init.cc:377] Logged in from keytab as oryx/127.25.236.188@KRBTEST.COM (short username oryx)
I20260806 01:14:32.043385 4285 webserver.cc:460] Webserver started at http://127.25.236.188:38121/ using document root <none> and password file <none>
I20260806 01:14:32.043865 4285 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:32.044014 4285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:32.047894 4285 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 01:14:32.050338 4597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:32.050784 4285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 01:14:32.050987 4285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/wal
uuid: "d446404a648949cea8e2c4eef9464be6"
format_stamp: "Formatted at 2026-08-06 01:14:31 on dist-test-slave-7nm7"
server_key: "7b97290a704db907e5c550ee7d8fb1d4"
server_key_iv: "ddc3d58637e093772759d24ec3c40a51"
server_key_version: "encryptionkey@0"
I20260806 01:14:32.051312 4285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:32.070986 4285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:32.076592 4285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:32.179312 4285 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.188:40513
I20260806 01:14:32.179342 4687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.188:40513 every 8 connection(s)
I20260806 01:14:32.180256 4285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/master-2/data/info.pb
I20260806 01:14:32.188472 4688 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:14:32.193979 4688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: Bootstrap starting.
I20260806 01:14:32.212229 4688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:14:32.212865 4688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: Bootstrap complete.
I20260806 01:14:32.213900 4688 raft_consensus.cc:348] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } }
I20260806 01:14:32.214434 4688 raft_consensus.cc:729] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d446404a648949cea8e2c4eef9464be6, State: Initialized, Role: LEARNER
I20260806 01:14:32.214784 4688 consensus_queue.cc:260] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } }
I20260806 01:14:32.215965 4693 sys_catalog.cc:455] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } } }
I20260806 01:14:32.216780 4688 sys_catalog.cc:564] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:32.216816 4693 sys_catalog.cc:458] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:14:32.229276 4707 catalog_manager.cc:1259] Loaded cluster ID: 48e3156596764ed7a8f5e71c7c835421
I20260806 01:14:32.229403 4707 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: loading cluster ID for follower catalog manager: success
I20260806 01:14:32.232462 4707 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: acquiring CA information for follower catalog manager: success
I20260806 01:14:32.235122 4707 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:14:32.288021 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:32.281189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:40513 (local address 127.0.0.1:45266)
0806 01:14:32.281470 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:32.281492 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:32.281620 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:32.282661 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:32.282667 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:32.282698 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:32.282917 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:32.282951 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.284496 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.284504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:32.285704 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:32.285717 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.287107 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.287117 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:32.287360 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:32.287403 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 01:14:32.287518 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 01:14:32.287679 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260806 01:14:32.288075 4710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:32.281437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45266 (local address 127.25.236.188:40513)
0806 01:14:32.282165 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:32.282169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:32.282188 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:14:32.282407 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:32.282412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:32.282492 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:32.282649 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:32.283071 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.284332 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.285882 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.286904 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.287137 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:32.287329 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:32.287547 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 01:14:32.287687 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":467,"threads_started":1}
I20260806 01:14:32.289499 4648 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 01:14:32.290123 4693 sys_catalog.cc:455] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0dac0217f664480abd32c52e2e36380. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } } }
I20260806 01:14:32.290467 4693 sys_catalog.cc:458] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: This master's current role is: LEARNER
W20260806 01:14:32.537889 4586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:32.538166 4586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:32.538398 4586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:32.546422 4586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:32.546542 4586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:32.546757 4586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:14:32.557956 4586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab
--principal=oryx/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:44437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=46879
--tserver_master_addrs=127.25.236.190:35167,127.25.236.189:34529,127.25.236.188:40513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:32.559477 4586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:32.561156 4586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:32.576709 4717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:32.577222 4719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:32.577968 4721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:32.578503 4586 server_base.cc:1034] running on GCE node
I20260806 01:14:32.579110 4586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:32.580188 4586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:32.581501 4586 hybrid_clock.cc:648] HybridClock initialized: now 1785978872581443 us; error 46 us; skew 500 ppm
I20260806 01:14:32.581878 4586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:32 dist-test-slave-7nm7 krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978872, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:32.585997 4586 init.cc:377] Logged in from keytab as oryx/127.25.236.129@KRBTEST.COM (short username oryx)
I20260806 01:14:32.587919 4586 webserver.cc:460] Webserver started at http://127.25.236.129:46879/ using document root <none> and password file <none>
I20260806 01:14:32.588672 4586 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:32.588876 4586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:32.595341 4586 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:14:32.598587 4730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:32.600226 4586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:14:32.600409 4586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "c1def5a943184a4aabe01c99379d51d2"
format_stamp: "Formatted at 2026-08-06 01:14:28 on dist-test-slave-7nm7"
server_key: "a1d76fcda5610d16037fb9564577e0ef"
server_key_iv: "af138e36d6a9f2abc06e1b5c29010e76"
server_key_version: "encryptionkey@0"
I20260806 01:14:32.601693 4586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:32.619098 4586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:32.624327 4586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:32.624666 4586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:32.625914 4586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:32.627750 4586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:32.627948 4586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:32.628106 4586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:32.628233 4586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:32.679307 4586 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:44437
I20260806 01:14:32.679329 4902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:44437 every 8 connection(s)
I20260806 01:14:32.680823 4586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:14:32.684798 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 4586
I20260806 01:14:32.685204 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 3710
Aug 06 01:14:32 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.236.130@KRBTEST.COM
I20260806 01:14:32.717420 4920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:32.686833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43568 (local address 127.25.236.130:36927)
0806 01:14:32.694798 (+ 7965us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:32.694805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:32.694831 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:14:32.694894 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:32.694900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:32.695172 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:32.697242 (+ 2070us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:32.697424 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.698630 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.700682 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.701767 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.703052 (+ 1285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:32.709587 (+ 6535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:32.709607 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:32.709626 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:32.709675 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:32.713548 (+ 3873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:32.714588 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:32.714597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:32.714600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:32.714754 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:32.715234 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:32.715241 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:32.715245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:32.715760 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:32.716157 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:32.716419 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 01:14:32.716811 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7818,"thread_start_us":7605,"threads_started":1}
Aug 06 01:14:32 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978872, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.189@KRBTEST.COM
Aug 06 01:14:32 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978872, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:32.718454 4563 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: NON_VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: true } } }
I20260806 01:14:32.718750 4563 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: This master's current role is: LEADER
Aug 06 01:14:32 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978872, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.188@KRBTEST.COM
I20260806 01:14:32.719517 3901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:32.720968 4929 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380: attempting to promote NON_VOTER d446404a648949cea8e2c4eef9464be6 to VOTER
I20260806 01:14:32.721828 4929 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } }
I20260806 01:14:32.724987 4648 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 1 LEARNER]: Refusing update from remote peer d0dac0217f664480abd32c52e2e36380: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 01:14:32.724996 4349 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 FOLLOWER]: Refusing update from remote peer d0dac0217f664480abd32c52e2e36380: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 01:14:32.725612 4563 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 01:14:32.726255 4563 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 01:14:32.730669 4710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:32.684440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:53755 (local address 127.25.236.188:40513)
0806 01:14:32.684615 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:32.684622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:32.700515 (+ 15893us) server_negotiation.cc:407] Connection header received
0806 01:14:32.703749 (+ 3234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:32.703754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:32.703842 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:32.703943 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:32.708886 (+ 4943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.710199 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.713347 (+ 3148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.714193 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.714354 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:32.720390 (+ 6036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:32.720397 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:32.720401 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:32.720444 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:32.725039 (+ 4595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:32.727557 (+ 2518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:32.727563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:32.727567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:32.727754 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:32.728389 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:32.728394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:32.728397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:32.728771 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:32.728912 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:32.729972 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0806 01:14:32.730299 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 01:14:32.732786 4925 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d446404a648949cea8e2c4eef9464be6 (127.25.236.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } } }
I20260806 01:14:32.734140 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:32.686348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:40513 (local address 127.25.236.129:53755)
0806 01:14:32.700383 (+ 14035us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:32.700407 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:32.703395 (+ 2988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:32.704885 (+ 1490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:32.704892 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:32.705850 (+ 958us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:32.707141 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:32.707172 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.710443 (+ 3271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.710448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:32.712210 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:32.712225 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.714665 (+ 2440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.714672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:32.714866 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:32.715498 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:32.715521 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:32.720208 (+ 4687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:32.725952 (+ 5744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:32.725962 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:32.725983 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:32.727410 (+ 1427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:32.727913 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:32.727918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:32.727921 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:32.728251 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:32.729027 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:32.729037 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:32.729498 (+ 461us) client_negotiation.cc:769] Sending connection context
0806 01:14:32.731165 (+ 1667us) client_negotiation.cc:240] Negotiation successful
0806 01:14:32.731785 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13861,"mutex_wait_us":26,"thread_start_us":14274,"threads_started":1}
I20260806 01:14:32.733769 4349 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d446404a648949cea8e2c4eef9464be6 (127.25.236.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } } }
I20260806 01:14:32.736420 4903 heartbeater.cc:347] Connected to a master server at 127.25.236.188:40513
I20260806 01:14:32.736747 4903 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:32.737711 4903 heartbeater.cc:511] Master 127.25.236.188:40513 requested a full tablet report, sending...
I20260806 01:14:32.737828 4563 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } } }
I20260806 01:14:32.738080 4563 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:32.738441 4563 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } } }
I20260806 01:14:32.738724 4563 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:32.741048 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:32.701170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.129:40415)
0806 01:14:32.702351 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:32.702369 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:32.703637 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:32.704725 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:32.704742 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:32.705803 (+ 1061us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:32.707158 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:32.707176 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.708830 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.708838 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:32.711573 (+ 2735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:32.711590 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.713215 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.713222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:32.713455 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:32.714141 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:32.714177 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:32.719595 (+ 5418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:32.732007 (+ 12412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:32.732014 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:32.732018 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:32.732398 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:32.738762 (+ 6364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:32.738767 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:32.738771 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:32.738974 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:32.739933 (+ 959us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:32.739941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:32.740127 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 01:14:32.740316 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 01:14:32.740632 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1030,"mutex_wait_us":217,"thread_start_us":754,"threads_started":1}
I20260806 01:14:32.741163 4923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:32.701312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:40415 (local address 127.25.236.190:35167)
0806 01:14:32.702933 (+ 1621us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:32.702939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:32.702960 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:14:32.704455 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:32.704460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:32.704533 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:32.704653 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:32.707371 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.708662 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.712376 (+ 3714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.712994 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.713130 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:32.726446 (+ 13316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:32.726454 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:32.726460 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:32.726510 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:32.731887 (+ 5377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:32.732523 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:32.732529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:32.732532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:32.737353 (+ 4821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:32.739100 (+ 1747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:32.739106 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:32.739110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:32.739546 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:32.739719 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:32.740545 (+ 826us) server_negotiation.cc:299] Negotiation successful
0806 01:14:32.740792 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1503,"thread_start_us":1351,"threads_started":1}
I20260806 01:14:32.742362 4905 heartbeater.cc:347] Connected to a master server at 127.25.236.190:35167
I20260806 01:14:32.742501 4905 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:32.743036 4905 heartbeater.cc:511] Master 127.25.236.190:35167 requested a full tablet report, sending...
I20260806 01:14:32.744392 3384 ts_manager.cc:194] Re-registered known tserver with Master: c1def5a943184a4aabe01c99379d51d2 (127.25.236.129:44437)
I20260806 01:14:32.745584 3384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:40415
I20260806 01:14:32.749745 4909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:32.685137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:54975 (local address 127.25.236.189:34529)
0806 01:14:32.685935 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:32.685941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:32.699136 (+ 13195us) server_negotiation.cc:407] Connection header received
0806 01:14:32.703747 (+ 4611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:32.703753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:32.703850 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:32.703983 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:32.707374 (+ 3391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.708625 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.711740 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.712461 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.712549 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:32.720494 (+ 7945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:32.720503 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:32.720508 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:32.720563 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:32.726596 (+ 6033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:32.746826 (+ 20230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:32.746836 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:32.746842 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:32.747053 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:32.747615 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:32.747623 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:32.747628 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:32.748055 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:32.748234 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:32.748885 (+ 651us) server_negotiation.cc:299] Negotiation successful
0806 01:14:32.749217 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":577,"threads_started":1}
I20260806 01:14:32.751114 4916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:32.685894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:34529 (local address 127.25.236.129:54975)
0806 01:14:32.698942 (+ 13048us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:32.699012 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:32.703395 (+ 4383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:32.704430 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:32.704442 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:32.705499 (+ 1057us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:32.707183 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:32.707194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.708838 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.708843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:32.711569 (+ 2726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:32.711585 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:32.712884 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:32.712891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:32.713178 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:32.714268 (+ 1090us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:32.714301 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:32.719223 (+ 4922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:32.726803 (+ 7580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:32.726823 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:32.726828 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:32.746623 (+ 19795us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:32.747244 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:32.747252 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:32.747258 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:32.747479 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:32.748330 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:32.748337 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:32.748556 (+ 219us) client_negotiation.cc:769] Sending connection context
0806 01:14:32.750029 (+ 1473us) client_negotiation.cc:240] Negotiation successful
0806 01:14:32.750513 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12686,"mutex_wait_us":263,"thread_start_us":12456,"threads_started":1}
I20260806 01:14:32.752209 4906 heartbeater.cc:347] Connected to a master server at 127.25.236.189:34529
I20260806 01:14:32.752350 4906 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:32.752854 4906 heartbeater.cc:511] Master 127.25.236.189:34529 requested a full tablet report, sending...
I20260806 01:14:32.753280 4932 sys_catalog.cc:455] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } } }
I20260806 01:14:32.753844 4932 sys_catalog.cc:458] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:14:32.755316 4330 ts_manager.cc:194] Registered new tserver with Master: c1def5a943184a4aabe01c99379d51d2 (127.25.236.129:44437)
I20260806 01:14:32.755795 4632 ts_manager.cc:194] Registered new tserver with Master: c1def5a943184a4aabe01c99379d51d2 (127.25.236.129:44437)
I20260806 01:14:32.756994 4925 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } } }
I20260806 01:14:32.757424 4925 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:32.757655 4648 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d446404a648949cea8e2c4eef9464be6 (127.25.236.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } } }
I20260806 01:14:32.761466 4693 sys_catalog.cc:455] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0dac0217f664480abd32c52e2e36380" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } } }
I20260806 01:14:32.761837 4693 sys_catalog.cc:458] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:14:32.833046 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 3710
I20260806 01:14:32.861545 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:36927
--local_ip_for_outbound_sockets=127.25.236.130
--tserver_master_addrs=127.25.236.190:35167
--webserver_port=33525
--webserver_interface=127.25.236.130
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab
--principal=oryx/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.236.190:35167,127.25.236.189:34529
--tserver_master_addrs=127.25.236.190:35167,127.25.236.189:34529,127.25.236.188:40513 with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:33.221055 4933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:33.221688 4933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:33.221961 4933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:33.231390 4933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:33.231520 4933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:33.231649 4933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:14:33.245012 4933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab
--principal=oryx/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:36927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=33525
--tserver_master_addrs=127.25.236.190:35167,127.25.236.189:34529,127.25.236.188:40513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:33.246716 4933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:33.248107 4933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:33.264542 4943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:33.264777 4949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:33.265036 4945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:33.266849 4933 server_base.cc:1034] running on GCE node
I20260806 01:14:33.267616 4933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:33.268855 4933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:33.270156 4933 hybrid_clock.cc:648] HybridClock initialized: now 1785978873270084 us; error 87 us; skew 500 ppm
I20260806 01:14:33.271085 4933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:33 dist-test-slave-7nm7 krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978873, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:33.275811 4933 init.cc:377] Logged in from keytab as oryx/127.25.236.130@KRBTEST.COM (short username oryx)
I20260806 01:14:33.277850 4933 webserver.cc:460] Webserver started at http://127.25.236.130:33525/ using document root <none> and password file <none>
I20260806 01:14:33.278825 4933 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:33.278959 4933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:33.285032 4933 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:14:33.288352 4959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:33.290983 4933 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260806 01:14:33.291209 4933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "e3691097671c4d49b2b6f1b7846272bf"
format_stamp: "Formatted at 2026-08-06 01:14:29 on dist-test-slave-7nm7"
server_key: "a12c7d6a0722da8da26c7d7743793f1b"
server_key_iv: "4d8a20c91c15adae19df18c82788fae2"
server_key_version: "encryptionkey@0"
I20260806 01:14:33.292640 4933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:33.305567 4933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:33.310007 4933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:33.310329 4933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:33.311384 4933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:33.312784 4933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:33.312871 4933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:33.313019 4933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:33.313112 4933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:33.347836 4933 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:36927
I20260806 01:14:33.347909 5135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:36927 every 8 connection(s)
I20260806 01:14:33.349251 4933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:14:33.353358 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 4933
I20260806 01:14:33.353708 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 3920
Aug 06 01:14:33 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978873, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.189@KRBTEST.COM
Aug 06 01:14:33 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.236.131@KRBTEST.COM
Aug 06 01:14:33 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978873, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
Aug 06 01:14:33 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978873, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.188@KRBTEST.COM
I20260806 01:14:33.396260 5149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:33.352467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:45355 (local address 127.25.236.189:34529)
0806 01:14:33.352985 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:33.352993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:33.356669 (+ 3676us) server_negotiation.cc:407] Connection header received
0806 01:14:33.368788 (+ 12119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:33.368796 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:33.368878 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:33.368962 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:33.372339 (+ 3377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.373694 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.377023 (+ 3329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.378002 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.378125 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:33.385787 (+ 7662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:33.385794 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:33.385798 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:33.385846 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:33.390814 (+ 4968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:33.392062 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:33.392081 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:33.392085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:33.392330 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:33.393108 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:33.393115 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:33.393124 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:33.393600 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:33.393847 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:33.395399 (+ 1552us) server_negotiation.cc:299] Negotiation successful
0806 01:14:33.395727 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":308,"threads_started":1}
I20260806 01:14:33.397261 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:33.353223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:34529 (local address 127.25.236.130:45355)
0806 01:14:33.356486 (+ 3263us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:33.356506 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:33.368395 (+ 11889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:33.369540 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:33.369554 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:33.370586 (+ 1032us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:33.372131 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:33.372157 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.373984 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.373990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:33.376832 (+ 2842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:33.376857 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.378637 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.378645 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:33.379140 (+ 495us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:33.379936 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:33.379974 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:33.385529 (+ 5555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:33.390964 (+ 5435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:33.390975 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:33.390998 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:33.391852 (+ 854us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:33.392512 (+ 660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:33.392517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:33.392529 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:33.392873 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:33.393929 (+ 1056us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:33.393939 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:33.394813 (+ 874us) client_negotiation.cc:769] Sending connection context
0806 01:14:33.395242 (+ 429us) client_negotiation.cc:240] Negotiation successful
0806 01:14:33.395822 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3126,"thread_start_us":1134,"threads_started":1}
I20260806 01:14:33.399178 5141 heartbeater.cc:347] Connected to a master server at 127.25.236.189:34529
I20260806 01:14:33.399631 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:33.354919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44550 (local address 127.25.236.131:41223)
0806 01:14:33.368441 (+ 13522us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:33.368450 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:33.368482 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:14:33.368557 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:33.368566 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:33.368842 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:33.368994 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:33.371612 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.373599 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.376152 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.377458 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.377606 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:33.387323 (+ 9717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:33.387344 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:33.387363 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:33.387408 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:33.393183 (+ 5775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:33.394505 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:33.394525 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:33.394536 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:33.394756 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:33.396063 (+ 1307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:33.396068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:33.396071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:33.396631 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:33.398499 (+ 1868us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:33.398669 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 01:14:33.399012 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13083,"thread_start_us":12916,"threads_started":1}
I20260806 01:14:33.399601 5141 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:33.401372 4106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:33.402164 5141 heartbeater.cc:511] Master 127.25.236.189:34529 requested a full tablet report, sending...
I20260806 01:14:33.403090 5154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:33.353273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:41307 (local address 127.25.236.190:35167)
0806 01:14:33.355829 (+ 2556us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:33.355838 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:33.355866 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:14:33.378499 (+ 22633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:33.378508 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:33.378629 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:33.378731 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:33.380455 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.381895 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.384550 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.385218 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.385330 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:33.390101 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:33.390109 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:33.390124 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:33.390181 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:33.395407 (+ 5226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:33.397729 (+ 2322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:33.397735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:33.397738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:33.398009 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:33.399032 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:33.399037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:33.399040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:33.399558 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:33.399744 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:33.400702 (+ 958us) server_negotiation.cc:299] Negotiation successful
0806 01:14:33.402545 (+ 1843us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2403,"thread_start_us":2277,"threads_started":1}
I20260806 01:14:33.404292 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:33.353089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.130:41307)
0806 01:14:33.354796 (+ 1707us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:33.354852 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:33.368282 (+ 13430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:33.379031 (+ 10749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:33.379039 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:33.379727 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:33.380199 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:33.380216 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.382098 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.382102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:33.384375 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:33.384390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.385668 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.385676 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:33.385865 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:33.386615 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:33.386639 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:33.389779 (+ 3140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:33.396557 (+ 6778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:33.396562 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:33.396566 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:33.397215 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:33.398692 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:33.398696 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:33.398699 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:33.398874 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:33.399809 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:33.399816 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:33.400038 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 01:14:33.400175 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 01:14:33.403647 (+ 3472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1375,"mutex_wait_us":1,"thread_start_us":2868,"threads_started":1}
I20260806 01:14:33.404727 4330 ts_manager.cc:194] Registered new tserver with Master: e3691097671c4d49b2b6f1b7846272bf (127.25.236.130:36927)
I20260806 01:14:33.405768 5138 heartbeater.cc:347] Connected to a master server at 127.25.236.190:35167
I20260806 01:14:33.405969 5138 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:33.406772 5138 heartbeater.cc:511] Master 127.25.236.190:35167 requested a full tablet report, sending...
I20260806 01:14:33.407311 5165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:33.357175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:40513 (local address 127.25.236.130:36067)
0806 01:14:33.368874 (+ 11699us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:33.368902 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:33.369062 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:33.382690 (+ 13628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:33.382694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:33.383679 (+ 985us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:33.384382 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:33.384399 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.387308 (+ 2909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.387316 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:33.389437 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:33.389455 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.391473 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.391485 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:33.391808 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:33.392552 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:33.392589 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:33.398217 (+ 5628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:33.403485 (+ 5268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:33.403491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:33.403498 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:33.404048 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:33.404795 (+ 747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:33.404800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:33.404803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:33.405018 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:33.405771 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:33.405776 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:33.405975 (+ 199us) client_negotiation.cc:769] Sending connection context
0806 01:14:33.406160 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 01:14:33.406571 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11548,"thread_start_us":11376,"threads_started":1}
I20260806 01:14:33.407980 3384 ts_manager.cc:194] Re-registered known tserver with Master: e3691097671c4d49b2b6f1b7846272bf (127.25.236.130:36927)
I20260806 01:14:33.408388 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:33.356767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:36067 (local address 127.25.236.188:40513)
0806 01:14:33.382136 (+ 25369us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:33.382143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:33.382173 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:14:33.382263 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:33.382269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:33.382359 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:33.382470 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:33.385785 (+ 3315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.387086 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.389619 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:33.390429 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:33.390568 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:33.398539 (+ 7971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:33.398546 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:33.398551 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:33.398595 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:33.403321 (+ 4726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:33.404216 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:33.404222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:33.404225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:33.404492 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:33.405143 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:33.405147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:33.405149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:33.405461 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:33.407656 (+ 2195us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:33.407804 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 01:14:33.408011 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25236,"thread_start_us":25104,"threads_started":1}
I20260806 01:14:33.409183 5136 heartbeater.cc:347] Connected to a master server at 127.25.236.188:40513
I20260806 01:14:33.409329 5136 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:33.409921 5136 heartbeater.cc:511] Master 127.25.236.188:40513 requested a full tablet report, sending...
I20260806 01:14:33.409845 3384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:41307
I20260806 01:14:33.411249 4632 ts_manager.cc:194] Registered new tserver with Master: e3691097671c4d49b2b6f1b7846272bf (127.25.236.130:36927)
I20260806 01:14:33.494107 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 3920
I20260806 01:14:33.518078 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:41223
--local_ip_for_outbound_sockets=127.25.236.131
--tserver_master_addrs=127.25.236.190:35167
--webserver_port=38181
--webserver_interface=127.25.236.131
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab
--principal=oryx/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:33065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.236.190:35167,127.25.236.189:34529
--tserver_master_addrs=127.25.236.190:35167,127.25.236.189:34529,127.25.236.188:40513 with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:14:33.748977 4905 heartbeater.cc:503] Master 127.25.236.190:35167 was elected leader, sending a full tablet report...
W20260806 01:14:33.875444 5167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:33.875735 5167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:33.875833 5167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:33.883569 5167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:33.883702 5167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:33.883821 5167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:14:33.894392 5167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:33065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab
--principal=oryx/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:41223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=38181
--tserver_master_addrs=127.25.236.190:35167,127.25.236.189:34529,127.25.236.188:40513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:33.895753 5167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:33.897596 5167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:33.913031 5180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:33.913105 5174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:33.913008 5176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:33.914318 5167 server_base.cc:1034] running on GCE node
I20260806 01:14:33.914978 5167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:33.916033 5167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:33.917294 5167 hybrid_clock.cc:648] HybridClock initialized: now 1785978873917173 us; error 97 us; skew 500 ppm
I20260806 01:14:33.917680 5167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:33 dist-test-slave-7nm7 krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978873, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:33.921057 5167 init.cc:377] Logged in from keytab as oryx/127.25.236.131@KRBTEST.COM (short username oryx)
I20260806 01:14:33.922778 5167 webserver.cc:460] Webserver started at http://127.25.236.131:38181/ using document root <none> and password file <none>
I20260806 01:14:33.923731 5167 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:33.923902 5167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:33.929850 5167 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 01:14:33.932711 5190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:33.934033 5167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:14:33.934280 5167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "7de709fd7b784db4997f104e2ca0f5a9"
format_stamp: "Formatted at 2026-08-06 01:14:30 on dist-test-slave-7nm7"
server_key: "1ebcd38274241ad86f6b14d6f6cbf919"
server_key_iv: "dcf630c6fd0c45cfa52754e779733c2e"
server_key_version: "encryptionkey@0"
I20260806 01:14:33.935338 5167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:33.958303 5167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:33.967350 5167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:33.967669 5167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:33.968771 5167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:33.970291 5167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:33.970423 5167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:33.970558 5167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:33.970646 5167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:34.007611 5167 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:41223
I20260806 01:14:34.007673 5356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:41223 every 8 connection(s)
I20260806 01:14:34.009168 5167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:14:34.016188 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 5167
Aug 06 01:14:34 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978873, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.189@KRBTEST.COM
Aug 06 01:14:34 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978873, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.188@KRBTEST.COM
Aug 06 01:14:34 dist-test-slave-7nm7 krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978873, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:34.069932 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.036876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.25.236.188:40513)
0806 01:14:34.040392 (+ 3516us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.040400 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.043481 (+ 3081us) server_negotiation.cc:407] Connection header received
0806 01:14:34.043557 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.043563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.043645 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.043753 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:34.046910 (+ 3157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.048528 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.051820 (+ 3292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.052918 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.053030 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.058605 (+ 5575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:34.058611 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:34.058616 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:34.058659 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:34.062983 (+ 4324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.064306 (+ 1323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.064313 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.064318 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.064479 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.066359 (+ 1880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.066368 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.066371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.066786 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:34.067053 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.069192 (+ 2139us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.069469 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3365,"thread_start_us":2652,"threads_started":1}
I20260806 01:14:34.081053 5371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.012856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:40513 (local address 127.25.236.131:44871)
0806 01:14:34.038024 (+ 25168us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:34.038116 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:34.044871 (+ 6755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:34.047364 (+ 2493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:34.047378 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:34.048217 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:34.049026 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.049050 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.057772 (+ 8722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.057778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.059616 (+ 1838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.059637 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.061052 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.061058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.061266 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.061862 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:34.061891 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:34.068929 (+ 7038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:34.074361 (+ 5432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:34.074381 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:34.074520 (+ 139us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:34.075603 (+ 1083us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:34.076176 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:34.076182 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:34.076186 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:34.076681 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:34.077597 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:34.077606 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:34.078292 (+ 686us) client_negotiation.cc:769] Sending connection context
0806 01:14:34.078645 (+ 353us) client_negotiation.cc:240] Negotiation successful
0806 01:14:34.079354 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24530,"mutex_wait_us":1,"thread_start_us":24260,"threads_started":1}
I20260806 01:14:34.081221 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.020376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.25.236.190:35167)
0806 01:14:34.041521 (+ 21145us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.041531 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.041560 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:14:34.041645 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.041654 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.041769 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.048177 (+ 6408us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:34.048357 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.049395 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.053391 (+ 3996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.054282 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.064011 (+ 9729us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.065052 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:34.065075 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:34.065083 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:34.065123 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:34.072058 (+ 6935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.077250 (+ 5192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.077274 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.077279 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.077454 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.077971 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.077977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.077980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.078438 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:34.078700 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.080370 (+ 1670us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.080609 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9570,"server-negotiator.queue_time_us":20963,"thread_start_us":22243,"threads_started":1}
I20260806 01:14:34.081964 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.013545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:34529 (local address 127.25.236.131:58569)
0806 01:14:34.042842 (+ 29297us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:34.042872 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:34.044882 (+ 2010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:34.049265 (+ 4383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:34.049270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:34.049882 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:34.050289 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.050303 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.055823 (+ 5520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.055830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.057427 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.057443 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.059810 (+ 2367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.059824 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.060113 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.061589 (+ 1476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:34.061632 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:34.066072 (+ 4440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:34.076614 (+ 10542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:34.076621 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:34.076628 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:34.077401 (+ 773us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:34.078671 (+ 1270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:34.078677 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:34.078683 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:34.078933 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:34.079608 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:34.079613 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:34.079771 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 01:14:34.079967 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 01:14:34.080269 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29117,"thread_start_us":28883,"threads_started":1}
I20260806 01:14:34.082171 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.012316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:44871 (local address 127.25.236.188:40513)
0806 01:14:34.039809 (+ 27493us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.039818 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.039847 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:14:34.045372 (+ 5525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.045379 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.045466 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.045571 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:34.051093 (+ 5522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.053186 (+ 2093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.059822 (+ 6636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.060869 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.060985 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.069170 (+ 8185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:34.069178 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:34.069182 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:34.069213 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:34.073951 (+ 4738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.075797 (+ 1846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.075806 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.075811 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.075983 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.076866 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.076874 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.076881 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.077336 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:34.077505 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.081569 (+ 4064us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.081815 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27329,"thread_start_us":28068,"threads_started":1}
I20260806 01:14:34.082249 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.046613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34396 (local address 127.25.236.189:34529)
0806 01:14:34.053170 (+ 6557us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.053176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.053195 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:14:34.053248 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.053255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.053438 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.055011 (+ 1573us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:34.057390 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.058766 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.065313 (+ 6547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.066025 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.071580 (+ 5555us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.071941 (+ 361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:34.071950 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:34.071957 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:34.072028 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:34.077817 (+ 5789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.078713 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.078718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.078721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.078981 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.079447 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.079451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.079454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.079770 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:34.080932 (+ 1162us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.081085 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.081766 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6404,"thread_start_us":6941,"threads_started":1}
I20260806 01:14:34.083720 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.012105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:58569 (local address 127.25.236.189:34529)
0806 01:14:34.046683 (+ 34578us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.046690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.046712 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:14:34.046776 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.046783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.046864 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.050454 (+ 3590us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:34.050599 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.053112 (+ 2513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.058910 (+ 5798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.059543 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.062930 (+ 3387us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.066799 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:34.066844 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:34.066851 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:34.066898 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:34.076430 (+ 9532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.078334 (+ 1904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.078339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.078343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.078506 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.079084 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.079091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.079097 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.079460 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:34.082958 (+ 3498us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.083080 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.083301 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34424,"thread_start_us":34270,"threads_started":1}
I20260806 01:14:34.084216 5357 heartbeater.cc:347] Connected to a master server at 127.25.236.188:40513
I20260806 01:14:34.084568 5357 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:34.084882 5362 heartbeater.cc:347] Connected to a master server at 127.25.236.189:34529
I20260806 01:14:34.085038 5362 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:34.086164 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.039012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:59951 (local address 127.25.236.190:35167)
0806 01:14:34.042605 (+ 3593us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.042614 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.043938 (+ 1324us) server_negotiation.cc:407] Connection header received
0806 01:14:34.051941 (+ 8003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.051947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.052029 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.052110 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:34.054385 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.055613 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.060481 (+ 4868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.061129 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.061281 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.069583 (+ 8302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:34.069588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:34.069592 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:34.069625 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:34.077067 (+ 7442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.079916 (+ 2849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.079924 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.079928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.080100 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.084291 (+ 4191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.084299 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.084303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.084680 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:34.084858 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.085488 (+ 630us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.085721 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":3457,"thread_start_us":2075,"threads_started":1}
I20260806 01:14:34.086760 5362 heartbeater.cc:511] Master 127.25.236.189:34529 requested a full tablet report, sending...
I20260806 01:14:34.086898 5357 heartbeater.cc:511] Master 127.25.236.188:40513 requested a full tablet report, sending...
I20260806 01:14:34.087181 5389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.041378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.131:59951)
0806 01:14:34.043256 (+ 1878us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:34.043280 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:34.051750 (+ 8470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:34.052388 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:34.052392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:34.053651 (+ 1259us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:34.054183 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.054198 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.055778 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.055786 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.059944 (+ 4158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.059957 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.063224 (+ 3267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.063232 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.063531 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.065666 (+ 2135us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:34.065691 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:34.068626 (+ 2935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:34.077531 (+ 8905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:34.077538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:34.077544 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:34.078121 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:34.080922 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:34.080927 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:34.080931 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:34.081743 (+ 812us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:34.084839 (+ 3096us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:34.084847 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:34.085135 (+ 288us) client_negotiation.cc:769] Sending connection context
0806 01:14:34.085316 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 01:14:34.085645 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1705,"thread_start_us":2050,"threads_started":1}
I20260806 01:14:34.088804 4330 ts_manager.cc:194] Registered new tserver with Master: 7de709fd7b784db4997f104e2ca0f5a9 (127.25.236.131:41223)
I20260806 01:14:34.088789 4632 ts_manager.cc:194] Registered new tserver with Master: 7de709fd7b784db4997f104e2ca0f5a9 (127.25.236.131:41223)
I20260806 01:14:34.089401 5359 heartbeater.cc:347] Connected to a master server at 127.25.236.190:35167
I20260806 01:14:34.089555 5359 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:34.090168 5359 heartbeater.cc:511] Master 127.25.236.190:35167 requested a full tablet report, sending...
I20260806 01:14:34.091272 3380 ts_manager.cc:194] Re-registered known tserver with Master: 7de709fd7b784db4997f104e2ca0f5a9 (127.25.236.131:41223)
I20260806 01:14:34.092549 3380 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:59951
I20260806 01:14:34.087329 3384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:14:34.098562 3384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:14:34.144588 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.123154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44566 (local address 127.25.236.131:41223)
0806 01:14:34.124141 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.124151 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.126876 (+ 2725us) server_negotiation.cc:407] Connection header received
0806 01:14:34.127264 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.127277 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.127633 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.127886 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:34.136192 (+ 8306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.138148 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.141564 (+ 3416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.143367 (+ 1803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.143589 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.143779 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.144066 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.144245 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":570,"threads_started":1}
I20260806 01:14:34.144767 5405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.122471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:41223 (local address 127.0.0.1:44566)
0806 01:14:34.126738 (+ 4267us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:34.126758 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:34.126893 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:34.128368 (+ 1475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:34.128375 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:34.128412 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:34.135743 (+ 7331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.135774 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.138303 (+ 2529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.138308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.141400 (+ 3092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.141415 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.143616 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.143622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.143832 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.143876 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 01:14:34.144013 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 01:14:34.144180 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4111,"mutex_wait_us":75,"thread_start_us":3980,"threads_started":1}
I20260806 01:14:34.145783 5407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.126816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:36927 (local address 127.0.0.1:43578)
0806 01:14:34.128919 (+ 2103us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:34.128945 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:34.129178 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:34.135505 (+ 6327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:34.135511 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:34.135547 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:34.136054 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.136069 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.138286 (+ 2217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.138294 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.140990 (+ 2696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.141006 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.143304 (+ 2298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.143319 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.143558 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.145216 (+ 1658us) client_negotiation.cc:769] Sending connection context
0806 01:14:34.145287 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 01:14:34.145418 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1911,"mutex_wait_us":75,"thread_start_us":1711,"threads_started":1}
I20260806 01:14:34.146867 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.127098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43578 (local address 127.25.236.130:36927)
0806 01:14:34.134763 (+ 7665us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.134770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.134797 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:14:34.134882 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.134888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.135097 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.135505 (+ 408us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:34.136308 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.138050 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.141210 (+ 3160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.142842 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.146090 (+ 3248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.146314 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.146384 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.146580 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7483,"thread_start_us":7358,"threads_started":1}
I20260806 01:14:34.148844 5409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.128861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:44437 (local address 127.0.0.1:60244)
0806 01:14:34.138631 (+ 9770us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:34.138673 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:34.138948 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:34.141870 (+ 2922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:34.141874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:34.141902 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:34.142322 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.142332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.143900 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.143906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.146398 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.146417 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.148071 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.148079 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.148275 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.148321 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 01:14:34.148462 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 01:14:34.148607 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9625,"thread_start_us":7015,"threads_started":1}
I20260806 01:14:34.149600 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.129128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60244 (local address 127.25.236.129:44437)
0806 01:14:34.141240 (+ 12112us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.141249 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.141279 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:14:34.141382 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.141390 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.141676 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.141890 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:34.142562 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.143769 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.146561 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.147825 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.148092 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.148269 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.149232 (+ 963us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.149370 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11940,"thread_start_us":11810,"threads_started":1}
I20260806 01:14:34.149762 5261 tablet_service.cc:1467] Processing CreateTablet for tablet cfc1f170625448f585526d4ba32b52ef (DEFAULT_TABLE table=test-table [id=cf938109f360465b92437c2ce0e67c85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:34.151430 5261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfc1f170625448f585526d4ba32b52ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:34.150915 5037 tablet_service.cc:1467] Processing CreateTablet for tablet cfc1f170625448f585526d4ba32b52ef (DEFAULT_TABLE table=test-table [id=cf938109f360465b92437c2ce0e67c85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:34.152292 5037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfc1f170625448f585526d4ba32b52ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:34.152887 4798 tablet_service.cc:1467] Processing CreateTablet for tablet cfc1f170625448f585526d4ba32b52ef (DEFAULT_TABLE table=test-table [id=cf938109f360465b92437c2ce0e67c85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:34.154277 4798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfc1f170625448f585526d4ba32b52ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:34.170115 5412 tablet_bootstrap.cc:492] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2: Bootstrap starting.
I20260806 01:14:34.171023 5413 tablet_bootstrap.cc:492] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf: Bootstrap starting.
I20260806 01:14:34.172012 5415 tablet_bootstrap.cc:492] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9: Bootstrap starting.
I20260806 01:14:34.176084 5415 tablet_bootstrap.cc:654] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:34.176421 5412 tablet_bootstrap.cc:654] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:34.177414 5415 log.cc:824] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:34.177903 5412 log.cc:824] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:34.181922 5413 tablet_bootstrap.cc:654] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:34.184521 5413 log.cc:824] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:34.189045 5415 tablet_bootstrap.cc:492] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9: No bootstrap required, opened a new log
I20260806 01:14:34.189054 5413 tablet_bootstrap.cc:492] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf: No bootstrap required, opened a new log
I20260806 01:14:34.189368 5415 ts_tablet_manager.cc:1397] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.000s
I20260806 01:14:34.189441 5413 ts_tablet_manager.cc:1397] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.000s
I20260806 01:14:34.199759 5412 tablet_bootstrap.cc:492] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2: No bootstrap required, opened a new log
I20260806 01:14:34.200105 5412 ts_tablet_manager.cc:1397] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2: Time spent bootstrapping tablet: real 0.030s user 0.014s sys 0.013s
I20260806 01:14:34.202931 5415 raft_consensus.cc:348] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:34.203557 5415 raft_consensus.cc:374] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:34.203747 5415 raft_consensus.cc:729] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7de709fd7b784db4997f104e2ca0f5a9, State: Initialized, Role: FOLLOWER
I20260806 01:14:34.203651 5413 raft_consensus.cc:348] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:34.204064 5413 raft_consensus.cc:374] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:34.204185 5413 raft_consensus.cc:729] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3691097671c4d49b2b6f1b7846272bf, State: Initialized, Role: FOLLOWER
I20260806 01:14:34.204802 5413 consensus_queue.cc:260] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:34.207053 5413 ts_tablet_manager.cc:1428] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf: Time spent starting tablet: real 0.017s user 0.012s sys 0.005s
I20260806 01:14:34.207366 5138 heartbeater.cc:503] Master 127.25.236.190:35167 was elected leader, sending a full tablet report...
I20260806 01:14:34.204802 5415 consensus_queue.cc:260] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:34.210839 5412 raft_consensus.cc:348] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:34.211305 5412 raft_consensus.cc:374] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:34.211407 5412 raft_consensus.cc:729] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1def5a943184a4aabe01c99379d51d2, State: Initialized, Role: FOLLOWER
I20260806 01:14:34.212137 5412 consensus_queue.cc:260] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:34.213747 5415 ts_tablet_manager.cc:1428] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9: Time spent starting tablet: real 0.024s user 0.010s sys 0.008s
I20260806 01:14:34.214046 5359 heartbeater.cc:503] Master 127.25.236.190:35167 was elected leader, sending a full tablet report...
I20260806 01:14:34.219182 5412 ts_tablet_manager.cc:1428] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2: Time spent starting tablet: real 0.019s user 0.010s sys 0.008s
W20260806 01:14:34.267624 5364 tablet.cc:2369] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:14:34.277575 5421 raft_consensus.cc:482] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:14:34.277874 5421 raft_consensus.cc:504] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:34.279956 5421 leader_election.cc:288] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7de709fd7b784db4997f104e2ca0f5a9 (127.25.236.131:41223), e3691097671c4d49b2b6f1b7846272bf (127.25.236.130:36927)
I20260806 01:14:34.288671 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.280449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:44091 (local address 127.25.236.131:41223)
0806 01:14:34.280770 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.280786 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.281227 (+ 441us) server_negotiation.cc:407] Connection header received
0806 01:14:34.281469 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.281477 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.281589 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.281700 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:34.282682 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.283615 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.286876 (+ 3261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.287703 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.287914 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.287995 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.288273 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.288407 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 01:14:34.288822 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.280213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:41223 (local address 127.25.236.129:44091)
0806 01:14:34.281079 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:34.281104 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:34.281306 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:34.281815 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:34.281819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:34.281846 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:34.282505 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.282521 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.283788 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.283795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.286705 (+ 2910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.286719 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.287929 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.287936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.288096 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.288125 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 01:14:34.288236 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 01:14:34.288380 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"mutex_wait_us":120,"thread_start_us":439,"threads_started":1}
I20260806 01:14:34.289561 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.281442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:53947 (local address 127.25.236.130:36927)
0806 01:14:34.281710 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.281716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.282151 (+ 435us) server_negotiation.cc:407] Connection header received
0806 01:14:34.282406 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.282410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.282470 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.282568 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:34.283079 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.283762 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.286913 (+ 3151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.287931 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.288324 (+ 393us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.288421 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.288743 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.288899 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260806 01:14:34.290036 5426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.281020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:36927 (local address 127.25.236.129:53947)
0806 01:14:34.282028 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:34.282046 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:34.282251 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:34.282589 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:34.282593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:34.282617 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:34.282925 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.282932 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.283919 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.283924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.286738 (+ 2814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:34.286752 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.288364 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.288374 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:34.288545 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.288579 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:14:34.288715 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 01:14:34.289533 (+ 818us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":45,"thread_start_us":569,"threads_started":1}
I20260806 01:14:34.290148 5286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc1f170625448f585526d4ba32b52ef" candidate_uuid: "c1def5a943184a4aabe01c99379d51d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7de709fd7b784db4997f104e2ca0f5a9" is_pre_election: true
I20260806 01:14:34.290917 5286 raft_consensus.cc:2454] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1def5a943184a4aabe01c99379d51d2 in term 0.
I20260806 01:14:34.291292 5074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc1f170625448f585526d4ba32b52ef" candidate_uuid: "c1def5a943184a4aabe01c99379d51d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3691097671c4d49b2b6f1b7846272bf" is_pre_election: true
I20260806 01:14:34.291963 5074 raft_consensus.cc:2454] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1def5a943184a4aabe01c99379d51d2 in term 0.
I20260806 01:14:34.291718 4732 leader_election.cc:302] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7de709fd7b784db4997f104e2ca0f5a9, c1def5a943184a4aabe01c99379d51d2; no voters:
I20260806 01:14:34.292713 5421 raft_consensus.cc:2781] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:14:34.292927 5421 raft_consensus.cc:482] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:14:34.293063 5421 raft_consensus.cc:3037] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:34.296306 5421 raft_consensus.cc:504] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:34.297050 5421 leader_election.cc:288] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [CANDIDATE]: Term 1 election: Requested vote from peers 7de709fd7b784db4997f104e2ca0f5a9 (127.25.236.131:41223), e3691097671c4d49b2b6f1b7846272bf (127.25.236.130:36927)
I20260806 01:14:34.297585 5074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc1f170625448f585526d4ba32b52ef" candidate_uuid: "c1def5a943184a4aabe01c99379d51d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3691097671c4d49b2b6f1b7846272bf"
I20260806 01:14:34.297601 5286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc1f170625448f585526d4ba32b52ef" candidate_uuid: "c1def5a943184a4aabe01c99379d51d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7de709fd7b784db4997f104e2ca0f5a9"
I20260806 01:14:34.297808 5074 raft_consensus.cc:3037] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:34.297809 5286 raft_consensus.cc:3037] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:34.300612 5074 raft_consensus.cc:2454] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1def5a943184a4aabe01c99379d51d2 in term 1.
I20260806 01:14:34.301077 4732 leader_election.cc:302] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1def5a943184a4aabe01c99379d51d2, e3691097671c4d49b2b6f1b7846272bf; no voters:
I20260806 01:14:34.301535 5421 raft_consensus.cc:2781] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:34.302109 5421 raft_consensus.cc:686] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [term 1 LEADER]: Becoming Leader. State: Replica: c1def5a943184a4aabe01c99379d51d2, State: Running, Role: LEADER
I20260806 01:14:34.302152 5286 raft_consensus.cc:2454] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1def5a943184a4aabe01c99379d51d2 in term 1.
I20260806 01:14:34.303130 5421 consensus_queue.cc:237] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:34.308928 3384 catalog_manager.cc:5495] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1def5a943184a4aabe01c99379d51d2 (127.25.236.129). New cstate: current_term: 1 leader_uuid: "c1def5a943184a4aabe01c99379d51d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } health_report { overall_health: HEALTHY } } }
I20260806 01:14:34.344079 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.337690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54044 (local address 127.25.236.129:44437)
0806 01:14:34.338008 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.338013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.338032 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:14:34.338351 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.338357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.338475 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.338617 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:14:34.339004 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.339965 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.341534 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.342092 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.342205 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.342766 (+ 561us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:14:34.343332 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:14:34.343524 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.343658 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.343725 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
W20260806 01:14:34.353283 5148 tablet.cc:2369] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:14:34.354139 5074 raft_consensus.cc:1261] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf [term 1 FOLLOWER]: Refusing update from remote peer c1def5a943184a4aabe01c99379d51d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:34.354758 5286 raft_consensus.cc:1261] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9 [term 1 FOLLOWER]: Refusing update from remote peer c1def5a943184a4aabe01c99379d51d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:34.355263 5421 consensus_queue.cc:1035] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:34.355654 5428 consensus_queue.cc:1035] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:34.397387 5437 mvcc.cc:205] Tried to move back new op lower bound from 7315369469342285824 to 7315369469150412800. Current Snapshot: MvccSnapshot[applied={T|T < 7315369469342285824}]
I20260806 01:14:34.416481 3384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:34.416915 3384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:34.427770 3384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0dac0217f664480abd32c52e2e36380: Sending DeleteTablet for 3 replicas of tablet cfc1f170625448f585526d4ba32b52ef
I20260806 01:14:34.429096 5261 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc1f170625448f585526d4ba32b52ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:34 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:44566
I20260806 01:14:34.429381 5037 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc1f170625448f585526d4ba32b52ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:34 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:43578
I20260806 01:14:34.430387 5445 tablet_replica.cc:330] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9: stopping tablet replica
I20260806 01:14:34.430941 5445 raft_consensus.cc:2229] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:34.431026 4798 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc1f170625448f585526d4ba32b52ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:34 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:60244
I20260806 01:14:34.431885 5445 raft_consensus.cc:2258] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:34.442204 5445 ts_tablet_manager.cc:1905] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:34.455240 5446 tablet_replica.cc:330] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf: stopping tablet replica
I20260806 01:14:34.455860 5446 raft_consensus.cc:2229] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:34.456708 5446 raft_consensus.cc:2258] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:34.460096 5447 tablet_replica.cc:330] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2: stopping tablet replica
I20260806 01:14:34.460654 5447 raft_consensus.cc:2229] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:14:34.461498 5447 raft_consensus.cc:2258] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:34.465181 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 3329
I20260806 01:14:34.466459 5445 ts_tablet_manager.cc:1918] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:34.466521 3438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:34.466691 5445 log.cc:1196] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/cfc1f170625448f585526d4ba32b52ef
I20260806 01:14:34.467326 5445 ts_tablet_manager.cc:1939] T cfc1f170625448f585526d4ba32b52ef P 7de709fd7b784db4997f104e2ca0f5a9: Deleting consensus metadata
I20260806 01:14:34.471824 5446 ts_tablet_manager.cc:1905] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:34.475615 5447 ts_tablet_manager.cc:1905] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:34.481927 5446 ts_tablet_manager.cc:1918] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:34.482262 5446 log.cc:1196] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/cfc1f170625448f585526d4ba32b52ef
I20260806 01:14:34.483086 5446 ts_tablet_manager.cc:1939] T cfc1f170625448f585526d4ba32b52ef P e3691097671c4d49b2b6f1b7846272bf: Deleting consensus metadata
I20260806 01:14:34.493630 5447 ts_tablet_manager.cc:1918] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:34.493870 5447 log.cc:1196] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/cfc1f170625448f585526d4ba32b52ef
I20260806 01:14:34.494746 5447 ts_tablet_manager.cc:1939] T cfc1f170625448f585526d4ba32b52ef P c1def5a943184a4aabe01c99379d51d2: Deleting consensus metadata
I20260806 01:14:34.499543 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.433629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38592 (local address 127.25.236.189:34529)
0806 01:14:34.433843 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.433849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.463681 (+ 29832us) server_negotiation.cc:407] Connection header received
0806 01:14:34.464003 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.464008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.464122 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.464202 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:34.472733 (+ 8531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.473991 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.478168 (+ 4177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.479580 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.479715 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.484578 (+ 4863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:34.484586 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:34.484591 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:34.484644 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:34.488820 (+ 4176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.496477 (+ 7657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.496484 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.496487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.496649 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.497137 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.497155 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.497161 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.497840 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:34.498160 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.498698 (+ 538us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.498971 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 01:14:34.519948 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.463836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37280 (local address 127.25.236.188:40513)
0806 01:14:34.464026 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.464033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.499415 (+ 35382us) server_negotiation.cc:407] Connection header received
0806 01:14:34.499492 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.499497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.499598 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.499698 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:34.502078 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.503634 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.505456 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.506576 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.506735 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.511043 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:34.511052 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:34.511059 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:34.511108 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:34.515577 (+ 4469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.516553 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.516560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.516566 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.516730 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.517218 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.517225 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.517232 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.517660 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:34.517921 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.519183 (+ 1262us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.519449 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:14:34.699769 3361 catalog_manager.cc:4841] TS 7de709fd7b784db4997f104e2ca0f5a9 (127.25.236.131:41223): tablet cfc1f170625448f585526d4ba32b52ef (table test-table [id=cf938109f360465b92437c2ce0e67c85]) successfully deleted
W20260806 01:14:34.700125 3361 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.25.236.131:41223, user_credentials={real_user=slave}} blocked reactor thread for 230713us
I20260806 01:14:34.700688 3361 catalog_manager.cc:4841] TS c1def5a943184a4aabe01c99379d51d2 (127.25.236.129:44437): tablet cfc1f170625448f585526d4ba32b52ef (table test-table [id=cf938109f360465b92437c2ce0e67c85]) successfully deleted
I20260806 01:14:34.701125 3361 catalog_manager.cc:4841] TS e3691097671c4d49b2b6f1b7846272bf (127.25.236.130:36927): tablet cfc1f170625448f585526d4ba32b52ef (table test-table [id=cf938109f360465b92437c2ce0e67c85]) successfully deleted
I20260806 01:14:34.704968 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 3329
I20260806 01:14:34.776947 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.755359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.25.236.189:34529)
0806 01:14:34.755544 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.755550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.756490 (+ 940us) server_negotiation.cc:407] Connection header received
0806 01:14:34.757379 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.757385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.757469 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.757543 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:34.760780 (+ 3237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.761938 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.764422 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.765096 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.765200 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.769927 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:34.769937 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:34.769942 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:34.769985 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:34.773566 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.774476 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.774484 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.774487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.774639 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.775125 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.775130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.775133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.775519 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:34.775692 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.776334 (+ 642us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.776567 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:14:34.779160 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:34.756256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37292 (local address 127.25.236.188:40513)
0806 01:14:34.756490 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:34.756499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:34.757039 (+ 540us) server_negotiation.cc:407] Connection header received
0806 01:14:34.757231 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:34.757240 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:34.757464 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:34.757635 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:34.763837 (+ 6202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.765513 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.767770 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:34.768217 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:34.768358 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:34.771471 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:34.771478 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:34.771483 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:34.771527 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:34.775291 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.776244 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.776250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.776253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.776416 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:34.776873 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:34.776878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:34.776882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:34.777405 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:34.778023 (+ 618us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:34.778185 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 01:14:34.778637 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:14:35.402935 5468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:35.402053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.129:44001)
0806 01:14:35.402636 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:35.402762 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":319,"threads_started":1}
W20260806 01:14:35.403448 4905 heartbeater.cc:644] Failed to heartbeat to 127.25.236.190:35167 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
I20260806 01:14:35.410377 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:35.409448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.131:34983)
0806 01:14:35.409977 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:35.410120 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":290,"threads_started":1}
W20260806 01:14:35.410795 5359 heartbeater.cc:644] Failed to heartbeat to 127.25.236.190:35167 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
I20260806 01:14:35.433691 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:35.432702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.130:36813)
0806 01:14:35.433345 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:35.433498 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":395,"threads_started":1}
W20260806 01:14:35.434155 5138 heartbeater.cc:644] Failed to heartbeat to 127.25.236.190:35167 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
I20260806 01:14:35.928351 5478 raft_consensus.cc:482] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d0dac0217f664480abd32c52e2e36380)
I20260806 01:14:35.928690 5478 raft_consensus.cc:504] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } }
I20260806 01:14:35.930330 5478 leader_election.cc:288] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0dac0217f664480abd32c52e2e36380 (127.25.236.190:35167), d446404a648949cea8e2c4eef9464be6 (127.25.236.188:40513)
I20260806 01:14:35.931491 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:35.930560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:38520)
0806 01:14:35.931063 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:35.931245 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":240,"threads_started":1}
I20260806 01:14:35.932822 5482 raft_consensus.cc:482] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d0dac0217f664480abd32c52e2e36380)
I20260806 01:14:35.933153 5482 raft_consensus.cc:504] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } }
I20260806 01:14:35.935601 5482 leader_election.cc:288] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0dac0217f664480abd32c52e2e36380 (127.25.236.190:35167), a703de63fe69433e876f4ef33496af84 (127.25.236.189:34529)
I20260806 01:14:35.936728 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:35.935713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:38532)
0806 01:14:35.936200 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:35.936424 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"threads_started":1}
W20260806 01:14:35.937139 4608 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
I20260806 01:14:35.945590 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:35.931275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:40513 (local address 127.0.0.1:37306)
0806 01:14:35.931681 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:35.931698 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:35.931826 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:35.932834 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:35.932839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:35.932865 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:35.933148 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:35.933176 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:35.934970 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:35.934976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:35.941428 (+ 6452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:35.941455 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:35.944016 (+ 2561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:35.944027 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:35.944506 (+ 479us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:35.944568 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 01:14:35.944708 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 01:14:35.944944 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"threads_started":1}
I20260806 01:14:35.947697 5481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:35.931784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37306 (local address 127.25.236.188:40513)
0806 01:14:35.932433 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:35.932448 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:35.932476 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:14:35.932580 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:35.932585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:35.932669 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:35.933416 (+ 747us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:35.933640 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:35.934797 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:35.941720 (+ 6923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:35.943413 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:35.946408 (+ 2995us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:35.946534 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:35.946623 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 01:14:35.946956 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":259,"threads_started":1}
I20260806 01:14:35.949074 5486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:35.939549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:38544)
0806 01:14:35.948650 (+ 9101us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:35.948856 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8912}
W20260806 01:14:35.950383 4608 leader_election.cc:334] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d0dac0217f664480abd32c52e2e36380 (127.25.236.190:35167): Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
W20260806 01:14:35.950893 4313 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
I20260806 01:14:35.952419 4648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a703de63fe69433e876f4ef33496af84" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d446404a648949cea8e2c4eef9464be6" is_pre_election: true
I20260806 01:14:35.952836 4648 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a703de63fe69433e876f4ef33496af84 in term 1.
I20260806 01:14:35.953434 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:35.952486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:38550)
0806 01:14:35.952709 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:35.952948 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 01:14:35.953610 4313 leader_election.cc:302] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a703de63fe69433e876f4ef33496af84, d446404a648949cea8e2c4eef9464be6; no voters:
I20260806 01:14:35.954130 5478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 01:14:35.954355 5478 raft_consensus.cc:482] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d0dac0217f664480abd32c52e2e36380)
I20260806 01:14:35.954493 5478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 1 FOLLOWER]: Advancing to term 2
W20260806 01:14:35.954149 4313 leader_election.cc:334] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d0dac0217f664480abd32c52e2e36380 (127.25.236.190:35167): Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
I20260806 01:14:35.954906 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:35.936104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:34529 (local address 127.0.0.1:38606)
0806 01:14:35.937010 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:35.937027 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:35.937153 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:35.944593 (+ 7440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:35.944601 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:35.944640 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:35.944942 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:35.944957 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:35.946928 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:35.946935 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:35.949958 (+ 3023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:35.949977 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:35.953448 (+ 3471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:35.953454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:35.953874 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:35.953913 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 01:14:35.953987 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 01:14:35.954391 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":12210,"threads_started":1}
I20260806 01:14:35.956262 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:35.936424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38606 (local address 127.25.236.189:34529)
0806 01:14:35.944179 (+ 7755us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:35.944187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:35.944211 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:14:35.944283 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:35.944294 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:35.944394 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:35.945113 (+ 719us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:35.945439 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:35.946726 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:35.950310 (+ 3584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:35.951915 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:35.955776 (+ 3861us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:35.955836 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:35.955879 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 01:14:35.955982 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7592,"thread_start_us":7420,"threads_started":1}
I20260806 01:14:35.956929 4349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d446404a648949cea8e2c4eef9464be6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a703de63fe69433e876f4ef33496af84" is_pre_election: true
I20260806 01:14:35.958391 5478 raft_consensus.cc:504] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } }
I20260806 01:14:35.958969 4349 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d446404a648949cea8e2c4eef9464be6 in current term 2: Already voted for candidate a703de63fe69433e876f4ef33496af84 in this term.
I20260806 01:14:35.959316 5478 leader_election.cc:288] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [CANDIDATE]: Term 2 election: Requested vote from peers d0dac0217f664480abd32c52e2e36380 (127.25.236.190:35167), d446404a648949cea8e2c4eef9464be6 (127.25.236.188:40513)
I20260806 01:14:35.959937 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:35.959425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:38556)
0806 01:14:35.959657 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:35.959781 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 01:14:35.960037 4648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a703de63fe69433e876f4ef33496af84" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d446404a648949cea8e2c4eef9464be6"
I20260806 01:14:35.960091 4600 leader_election.cc:302] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d446404a648949cea8e2c4eef9464be6; no voters: a703de63fe69433e876f4ef33496af84, d0dac0217f664480abd32c52e2e36380
I20260806 01:14:35.960320 4648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:14:35.961360 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:35.960853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:38558)
0806 01:14:35.961012 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:35.961097 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 01:14:35.961549 4313 leader_election.cc:334] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d0dac0217f664480abd32c52e2e36380 (127.25.236.190:35167): Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
I20260806 01:14:35.963224 4648 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a703de63fe69433e876f4ef33496af84 in term 2.
I20260806 01:14:35.963526 5482 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 01:14:35.963891 4313 leader_election.cc:302] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a703de63fe69433e876f4ef33496af84, d446404a648949cea8e2c4eef9464be6; no voters: d0dac0217f664480abd32c52e2e36380
I20260806 01:14:35.964208 5478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:14:35.964716 5478 raft_consensus.cc:686] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [term 2 LEADER]: Becoming Leader. State: Replica: a703de63fe69433e876f4ef33496af84, State: Running, Role: LEADER
I20260806 01:14:35.965152 5478 consensus_queue.cc:237] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } }
I20260806 01:14:35.968678 5492 sys_catalog.cc:455] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a703de63fe69433e876f4ef33496af84. Latest consensus state: current_term: 2 leader_uuid: "a703de63fe69433e876f4ef33496af84" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } } }
I20260806 01:14:35.969153 5492 sys_catalog.cc:458] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:35.970306 5497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:35.973897 5497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:35.974792 5497 catalog_manager.cc:1259] Loaded cluster ID: 48e3156596764ed7a8f5e71c7c835421
I20260806 01:14:35.974861 5497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:35.976047 5497 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:35.977248 5497 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: Loaded TSK: 0
I20260806 01:14:35.978210 5497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:14:36.025352 4329 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:14:36.041009 4329 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:14:36.046612 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.046185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:38596)
0806 01:14:36.046353 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.046456 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 01:14:36.047259 4648 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [term 2 FOLLOWER]: Refusing update from remote peer a703de63fe69433e876f4ef33496af84: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 01:14:36.047619 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.047262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:38606)
0806 01:14:36.047415 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.047505 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 01:14:36.047886 4313 consensus_peers.cc:487] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 -> Peer d0dac0217f664480abd32c52e2e36380 (127.25.236.190:35167): Couldn't send request to peer d0dac0217f664480abd32c52e2e36380. Status: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 01:14:36.048328 5492 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 01:14:36.051854 5482 sys_catalog.cc:455] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a703de63fe69433e876f4ef33496af84. Latest consensus state: current_term: 2 leader_uuid: "a703de63fe69433e876f4ef33496af84" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } } }
I20260806 01:14:36.052168 5482 sys_catalog.cc:458] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:14:36.053987 5478 sys_catalog.cc:455] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a703de63fe69433e876f4ef33496af84" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } } }
I20260806 01:14:36.054700 5478 sys_catalog.cc:458] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:36.055757 5482 sys_catalog.cc:455] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a703de63fe69433e876f4ef33496af84" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0dac0217f664480abd32c52e2e36380" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35167 } } peers { permanent_uuid: "a703de63fe69433e876f4ef33496af84" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 34529 } attrs { promote: false } } peers { permanent_uuid: "d446404a648949cea8e2c4eef9464be6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 40513 } attrs { promote: false } } }
I20260806 01:14:36.056082 5482 sys_catalog.cc:458] T 00000000000000000000000000000000 P d446404a648949cea8e2c4eef9464be6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:14:36.073215 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.065818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:41223 (local address 127.0.0.1:52908)
0806 01:14:36.066018 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.066041 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:36.066309 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:36.066982 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:36.066990 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:36.067031 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:36.067584 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:36.067600 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.069218 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.069222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:36.070702 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:36.070716 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.072380 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.072387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:36.072562 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.072598 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 01:14:36.072675 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 01:14:36.072803 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 01:14:36.074723 5508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.065937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52908 (local address 127.25.236.131:41223)
0806 01:14:36.066571 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:36.066577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:36.066597 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:14:36.066674 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:36.066682 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:36.066788 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:36.066900 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:36.067756 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.068986 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.070910 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.072151 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.073864 (+ 1713us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.073949 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:36.074004 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 01:14:36.074274 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":398,"threads_started":1}
I20260806 01:14:36.075670 5261 tablet_service.cc:1467] Processing CreateTablet for tablet c0c592f726664c3d996437a99397c35d (DEFAULT_TABLE table=test-table [id=604bcf4e52004e35b226e169a1a22ca1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:36.076176 5261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0c592f726664c3d996437a99397c35d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:36.077755 5510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.066637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:44437 (local address 127.0.0.1:54050)
0806 01:14:36.068230 (+ 1593us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.068253 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:36.068387 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:36.070403 (+ 2016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:36.070407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:36.070433 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:36.071343 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:36.071357 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.072925 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.072931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:36.075233 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:36.075251 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.076863 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.076870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:36.077044 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.077081 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:14:36.077272 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 01:14:36.077419 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1435,"thread_start_us":1305,"threads_started":1}
I20260806 01:14:36.078998 5512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.066787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54050 (local address 127.25.236.129:44437)
0806 01:14:36.068817 (+ 2030us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:36.068822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:36.068836 (+ 14us) server_negotiation.cc:407] Connection header received
0806 01:14:36.068920 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:36.068925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:36.069003 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:36.071511 (+ 2508us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:36.071711 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.072783 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.075461 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.076607 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.078038 (+ 1431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.078123 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:36.078490 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 01:14:36.078661 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1930,"thread_start_us":1829,"threads_started":1}
I20260806 01:14:36.079825 4798 tablet_service.cc:1467] Processing CreateTablet for tablet c0c592f726664c3d996437a99397c35d (DEFAULT_TABLE table=test-table [id=604bcf4e52004e35b226e169a1a22ca1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:36.080286 4798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0c592f726664c3d996437a99397c35d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:36.084501 5514 tablet_bootstrap.cc:492] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9: Bootstrap starting.
I20260806 01:14:36.085400 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.066332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:36927 (local address 127.0.0.1:49188)
0806 01:14:36.067261 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.067296 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:36.067507 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:36.078642 (+ 11135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:36.078646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:36.078669 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:36.078786 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:36.078793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.081067 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.081072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:36.082939 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:36.082954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.084706 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.084712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:36.084963 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.085001 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 01:14:36.085071 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 01:14:36.085181 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652}
I20260806 01:14:36.086364 5516 tablet_bootstrap.cc:492] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2: Bootstrap starting.
I20260806 01:14:36.086488 5514 tablet_bootstrap.cc:654] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:36.086549 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.067796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49188 (local address 127.25.236.130:36927)
0806 01:14:36.077945 (+ 10149us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:36.077952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:36.077973 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:14:36.078050 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:36.078054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:36.078115 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:36.078340 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:36.078932 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.080883 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.083121 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.084454 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.085694 (+ 1240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.085791 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:36.085856 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 01:14:36.086024 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10016,"thread_start_us":9899,"threads_started":1}
I20260806 01:14:36.087606 5037 tablet_service.cc:1467] Processing CreateTablet for tablet c0c592f726664c3d996437a99397c35d (DEFAULT_TABLE table=test-table [id=604bcf4e52004e35b226e169a1a22ca1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:36.088054 5037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0c592f726664c3d996437a99397c35d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:36.088318 5516 tablet_bootstrap.cc:654] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:36.088918 5514 tablet_bootstrap.cc:492] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9: No bootstrap required, opened a new log
I20260806 01:14:36.089068 5514 ts_tablet_manager.cc:1397] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 01:14:36.089998 5514 raft_consensus.cc:348] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:36.090219 5514 raft_consensus.cc:374] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:36.090438 5514 raft_consensus.cc:729] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7de709fd7b784db4997f104e2ca0f5a9, State: Initialized, Role: FOLLOWER
I20260806 01:14:36.090691 5516 tablet_bootstrap.cc:492] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2: No bootstrap required, opened a new log
I20260806 01:14:36.090830 5516 ts_tablet_manager.cc:1397] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 01:14:36.090685 5514 consensus_queue.cc:260] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:36.092157 5514 ts_tablet_manager.cc:1428] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 01:14:36.095085 5362 heartbeater.cc:503] Master 127.25.236.189:34529 was elected leader, sending a full tablet report...
I20260806 01:14:36.096120 5522 tablet_bootstrap.cc:492] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf: Bootstrap starting.
I20260806 01:14:36.097892 5516 raft_consensus.cc:348] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:36.098033 5522 tablet_bootstrap.cc:654] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:36.098124 5516 raft_consensus.cc:374] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:36.098264 5516 raft_consensus.cc:729] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1def5a943184a4aabe01c99379d51d2, State: Initialized, Role: FOLLOWER
I20260806 01:14:36.098524 5516 consensus_queue.cc:260] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:36.100435 5522 tablet_bootstrap.cc:492] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf: No bootstrap required, opened a new log
I20260806 01:14:36.100593 5522 ts_tablet_manager.cc:1397] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 01:14:36.100802 5516 ts_tablet_manager.cc:1428] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
I20260806 01:14:36.101435 5529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.099856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.129:56953)
0806 01:14:36.100903 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.101131 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":673,"threads_started":1}
I20260806 01:14:36.102730 5522 raft_consensus.cc:348] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:36.103061 5522 raft_consensus.cc:374] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:36.103183 5522 raft_consensus.cc:729] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3691097671c4d49b2b6f1b7846272bf, State: Initialized, Role: FOLLOWER
I20260806 01:14:36.103484 5522 consensus_queue.cc:260] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:36.104590 4906 heartbeater.cc:503] Master 127.25.236.189:34529 was elected leader, sending a full tablet report...
I20260806 01:14:36.107273 5529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.102020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.129:55993)
0806 01:14:36.106814 (+ 4794us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.106944 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4659,"threads_started":1}
I20260806 01:14:36.107944 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.091847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.131:51499)
0806 01:14:36.107656 (+ 15809us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.107767 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15672,"threads_started":1}
I20260806 01:14:36.109437 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.109053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.131:32965)
0806 01:14:36.109207 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.109300 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 01:14:36.109740 5522 ts_tablet_manager.cc:1428] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260806 01:14:36.112501 5141 heartbeater.cc:503] Master 127.25.236.189:34529 was elected leader, sending a full tablet report...
I20260806 01:14:36.113878 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.110385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.130:37745)
0806 01:14:36.113540 (+ 3155us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.113681 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3015,"thread_start_us":2102,"threads_started":1}
I20260806 01:14:36.114872 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.114498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.130:57609)
0806 01:14:36.114630 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.114733 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 01:14:36.131914 5532 raft_consensus.cc:482] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:14:36.132177 5532 raft_consensus.cc:504] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:36.134028 5532 leader_election.cc:288] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7de709fd7b784db4997f104e2ca0f5a9 (127.25.236.131:41223), c1def5a943184a4aabe01c99379d51d2 (127.25.236.129:44437)
I20260806 01:14:36.140301 5508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.134396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:35643 (local address 127.25.236.131:41223)
0806 01:14:36.134566 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:36.134574 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:36.134597 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:14:36.134926 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:36.134934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:36.135026 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:36.135093 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:36.135798 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.136523 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.138168 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.139120 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.139401 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.139482 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:36.139906 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 01:14:36.140035 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 01:14:36.140354 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.134266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:41223 (local address 127.25.236.130:35643)
0806 01:14:36.134515 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.134537 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:36.134710 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:36.135316 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:36.135321 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:36.135356 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:36.135642 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:36.135651 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.136670 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.136679 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:36.138001 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:36.138013 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.139414 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.139420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:36.139673 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.139707 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:14:36.139861 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 01:14:36.140011 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":25}
I20260806 01:14:36.141147 5286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0c592f726664c3d996437a99397c35d" candidate_uuid: "e3691097671c4d49b2b6f1b7846272bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7de709fd7b784db4997f104e2ca0f5a9" is_pre_election: true
I20260806 01:14:36.141431 5286 raft_consensus.cc:2454] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3691097671c4d49b2b6f1b7846272bf in term 0.
I20260806 01:14:36.142032 4964 leader_election.cc:302] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7de709fd7b784db4997f104e2ca0f5a9, e3691097671c4d49b2b6f1b7846272bf; no voters:
I20260806 01:14:36.142535 5532 raft_consensus.cc:2781] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:14:36.142706 5532 raft_consensus.cc:482] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:14:36.142776 5512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.134869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:51205 (local address 127.25.236.129:44437)
0806 01:14:36.135170 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:36.135193 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:36.135590 (+ 397us) server_negotiation.cc:407] Connection header received
0806 01:14:36.135811 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:36.135815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:36.135874 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:36.135975 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:36.136822 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.137816 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.140381 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.141482 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.141665 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.141751 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:36.142345 (+ 594us) server_negotiation.cc:299] Negotiation successful
0806 01:14:36.142506 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 01:14:36.142832 5532 raft_consensus.cc:3037] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:36.142894 5535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.134610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:44437 (local address 127.25.236.130:51205)
0806 01:14:36.135482 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.135498 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:36.135665 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:36.135987 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:36.135991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:36.136014 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:36.136643 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:36.136653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.138196 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.138203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:36.140186 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:36.140208 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.141832 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.141838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:36.142053 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.142090 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:14:36.142220 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 01:14:36.142374 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":23,"thread_start_us":613,"threads_started":1}
I20260806 01:14:36.143878 4834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0c592f726664c3d996437a99397c35d" candidate_uuid: "e3691097671c4d49b2b6f1b7846272bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1def5a943184a4aabe01c99379d51d2" is_pre_election: true
I20260806 01:14:36.144308 4834 raft_consensus.cc:2454] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3691097671c4d49b2b6f1b7846272bf in term 0.
I20260806 01:14:36.145493 5532 raft_consensus.cc:504] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:36.146039 5532 leader_election.cc:288] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [CANDIDATE]: Term 1 election: Requested vote from peers 7de709fd7b784db4997f104e2ca0f5a9 (127.25.236.131:41223), c1def5a943184a4aabe01c99379d51d2 (127.25.236.129:44437)
I20260806 01:14:36.146680 5286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0c592f726664c3d996437a99397c35d" candidate_uuid: "e3691097671c4d49b2b6f1b7846272bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7de709fd7b784db4997f104e2ca0f5a9"
I20260806 01:14:36.146876 5286 raft_consensus.cc:3037] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:36.146934 4834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0c592f726664c3d996437a99397c35d" candidate_uuid: "e3691097671c4d49b2b6f1b7846272bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1def5a943184a4aabe01c99379d51d2"
I20260806 01:14:36.147167 4834 raft_consensus.cc:3037] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:36.149124 5286 raft_consensus.cc:2454] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3691097671c4d49b2b6f1b7846272bf in term 1.
I20260806 01:14:36.149466 4834 raft_consensus.cc:2454] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3691097671c4d49b2b6f1b7846272bf in term 1.
I20260806 01:14:36.149576 4964 leader_election.cc:302] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7de709fd7b784db4997f104e2ca0f5a9, e3691097671c4d49b2b6f1b7846272bf; no voters:
I20260806 01:14:36.149957 5532 raft_consensus.cc:2781] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:36.150619 5532 raft_consensus.cc:686] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [term 1 LEADER]: Becoming Leader. State: Replica: e3691097671c4d49b2b6f1b7846272bf, State: Running, Role: LEADER
I20260806 01:14:36.150985 5532 consensus_queue.cc:237] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } }
I20260806 01:14:36.152318 5535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.151810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.130:55227)
0806 01:14:36.152022 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.152155 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260806 01:14:36.154698 4330 catalog_manager.cc:5495] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e3691097671c4d49b2b6f1b7846272bf (127.25.236.130). New cstate: current_term: 1 leader_uuid: "e3691097671c4d49b2b6f1b7846272bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3691097671c4d49b2b6f1b7846272bf" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 36927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 } health_report { overall_health: UNKNOWN } } }
I20260806 01:14:36.183804 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.178050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49202 (local address 127.25.236.130:36927)
0806 01:14:36.178319 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:36.178324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:36.178341 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:14:36.178432 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:36.178435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:36.178522 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:36.178646 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:14:36.179085 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.179784 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.181650 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.182043 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.182144 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.182674 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:14:36.183197 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:14:36.183309 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:36.183427 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 01:14:36.183494 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:14:36.188361 5286 raft_consensus.cc:1261] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9 [term 1 FOLLOWER]: Refusing update from remote peer e3691097671c4d49b2b6f1b7846272bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:36.188609 4834 raft_consensus.cc:1261] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2 [term 1 FOLLOWER]: Refusing update from remote peer e3691097671c4d49b2b6f1b7846272bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 01:14:36.189311 4908 tablet.cc:2369] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:14:36.189100 5532 consensus_queue.cc:1035] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7de709fd7b784db4997f104e2ca0f5a9" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 41223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 01:14:36.189842 5536 consensus_queue.cc:1035] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1def5a943184a4aabe01c99379d51d2" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 44437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:36.193224 5530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.192539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.129:51453)
0806 01:14:36.192791 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.193003 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 01:14:36.193346 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.192667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.131:33783)
0806 01:14:36.192946 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.193065 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260806 01:14:36.202755 5535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.202219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.130:55535)
0806 01:14:36.202420 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.202519 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 01:14:36.205175 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.204688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.131:37871)
0806 01:14:36.204872 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.204972 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 01:14:36.206213 5543 mvcc.cc:205] Tried to move back new op lower bound from 7315369476859629568 to 7315369476719112192. Current Snapshot: MvccSnapshot[applied={T|T < 7315369476859629568 or (T in {7315369476859629568})}]
I20260806 01:14:36.209036 5535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.208583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.130:58089)
0806 01:14:36.208758 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.208883 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 01:14:36.211679 5530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.211229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.25.236.129:56499)
0806 01:14:36.211446 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.211535 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260806 01:14:36.215956 4330 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:36.216218 4330 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:36.222441 4330 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a703de63fe69433e876f4ef33496af84: Sending DeleteTablet for 3 replicas of tablet c0c592f726664c3d996437a99397c35d
I20260806 01:14:36.223757 5261 tablet_service.cc:1514] Processing DeleteTablet for tablet c0c592f726664c3d996437a99397c35d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:36 UTC) from {username='oryx', principal='oryx/127.25.236.189@KRBTEST.COM'} at 127.0.0.1:52908
I20260806 01:14:36.224097 5037 tablet_service.cc:1514] Processing DeleteTablet for tablet c0c592f726664c3d996437a99397c35d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:36 UTC) from {username='oryx', principal='oryx/127.25.236.189@KRBTEST.COM'} at 127.0.0.1:49188
I20260806 01:14:36.224659 5554 tablet_replica.cc:330] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9: stopping tablet replica
I20260806 01:14:36.224927 5554 raft_consensus.cc:2229] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:36.225207 4798 tablet_service.cc:1514] Processing DeleteTablet for tablet c0c592f726664c3d996437a99397c35d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:36 UTC) from {username='oryx', principal='oryx/127.25.236.189@KRBTEST.COM'} at 127.0.0.1:54050
I20260806 01:14:36.225256 5554 raft_consensus.cc:2258] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:36.225481 4797 tablet_service.cc:1514] Processing DeleteTablet for tablet c0c592f726664c3d996437a99397c35d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:36 UTC) from {username='oryx', principal='oryx/127.25.236.189@KRBTEST.COM'} at 127.0.0.1:54050
I20260806 01:14:36.226002 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 4586
I20260806 01:14:36.226027 5556 tablet_replica.cc:330] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2: stopping tablet replica
I20260806 01:14:36.226425 5556 raft_consensus.cc:2229] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:36.226579 5554 ts_tablet_manager.cc:1905] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:36.226936 5556 raft_consensus.cc:2258] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:36.228443 5556 ts_tablet_manager.cc:1905] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:36.234719 5555 tablet_replica.cc:330] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf: stopping tablet replica
I20260806 01:14:36.234953 5555 raft_consensus.cc:2229] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:14:36.235074 5554 ts_tablet_manager.cc:1918] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:36.235234 5554 log.cc:1196] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/c0c592f726664c3d996437a99397c35d
I20260806 01:14:36.235615 5555 raft_consensus.cc:2258] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:36.235714 5554 ts_tablet_manager.cc:1939] T c0c592f726664c3d996437a99397c35d P 7de709fd7b784db4997f104e2ca0f5a9: Deleting consensus metadata
I20260806 01:14:36.236716 5555 ts_tablet_manager.cc:1905] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:36.236994 4311 catalog_manager.cc:4841] TS 7de709fd7b784db4997f104e2ca0f5a9 (127.25.236.131:41223): tablet c0c592f726664c3d996437a99397c35d (table test-table [id=604bcf4e52004e35b226e169a1a22ca1]) successfully deleted
I20260806 01:14:36.238685 5556 ts_tablet_manager.cc:1918] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:36.238942 5556 log.cc:1196] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/c0c592f726664c3d996437a99397c35d
I20260806 01:14:36.239516 5556 ts_tablet_manager.cc:1939] T c0c592f726664c3d996437a99397c35d P c1def5a943184a4aabe01c99379d51d2: Deleting consensus metadata
I20260806 01:14:36.240815 4311 catalog_manager.cc:4841] TS c1def5a943184a4aabe01c99379d51d2 (127.25.236.129:44437): tablet c0c592f726664c3d996437a99397c35d (table test-table [id=604bcf4e52004e35b226e169a1a22ca1]) successfully deleted
W20260806 01:14:36.241262 4311 catalog_manager.cc:4808] TS c1def5a943184a4aabe01c99379d51d2 (127.25.236.129:44437): delete failed for tablet c0c592f726664c3d996437a99397c35d because the tablet was not found. No further retry: Not found: Tablet not found: c0c592f726664c3d996437a99397c35d
I20260806 01:14:36.242956 5555 ts_tablet_manager.cc:1918] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:36.243108 5555 log.cc:1196] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/c0c592f726664c3d996437a99397c35d
I20260806 01:14:36.243649 5555 ts_tablet_manager.cc:1939] T c0c592f726664c3d996437a99397c35d P e3691097671c4d49b2b6f1b7846272bf: Deleting consensus metadata
I20260806 01:14:36.244791 4311 catalog_manager.cc:4841] TS e3691097671c4d49b2b6f1b7846272bf (127.25.236.130:36927): tablet c0c592f726664c3d996437a99397c35d (table test-table [id=604bcf4e52004e35b226e169a1a22ca1]) successfully deleted
I20260806 01:14:36.254344 5512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.228405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.25.236.129:44437)
0806 01:14:36.228573 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:36.228580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:36.241033 (+ 12453us) server_negotiation.cc:407] Connection header received
0806 01:14:36.241492 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:36.241507 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:36.241569 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:36.241624 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:36.243136 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.243978 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.245303 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.245691 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.245830 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.248366 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:36.248378 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:36.248392 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:36.248421 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:36.251337 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:36.252095 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:36.252102 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:36.252104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:36.252226 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:36.252595 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:36.252604 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:36.252606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:36.253122 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:36.253430 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:36.253732 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 01:14:36.253942 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:14:36.255056 4897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:36.380815 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 4586
I20260806 01:14:36.424070 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 4933
I20260806 01:14:36.441285 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.424941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49206 (local address 127.25.236.130:36927)
0806 01:14:36.425153 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:36.425159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:36.425183 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:14:36.425508 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:36.425515 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:36.425598 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:36.425715 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:36.427187 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.428400 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.429632 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.430320 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.430449 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.433126 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:36.433143 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:36.433166 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:36.433212 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:36.437458 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:36.438160 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:36.438170 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:36.438174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:36.438454 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:36.439095 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:36.439103 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:36.439108 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:36.439785 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:36.440217 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:36.440470 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 01:14:36.440774 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 01:14:36.442679 5130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:36.524407 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.523929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:38612)
0806 01:14:36.524146 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.524239 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 01:14:36.525449 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.525100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35167 (local address 127.0.0.1:38626)
0806 01:14:36.525247 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:36.525325 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 01:14:36.568118 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 4933
I20260806 01:14:36.598342 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 5167
I20260806 01:14:36.614545 5508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:36.599166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52912 (local address 127.25.236.131:41223)
0806 01:14:36.599350 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:36.599355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:36.599373 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:14:36.599440 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:36.599444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:36.599507 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:36.599601 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:36.602003 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.603073 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.604456 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:36.604885 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:36.605026 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:36.607980 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:36.608006 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:36.608023 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:36.608069 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:36.611003 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:36.611783 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:36.611791 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:36.611793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:36.611926 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:36.612433 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:36.612437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:36.612439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:36.612819 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:36.613055 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:36.613891 (+ 836us) server_negotiation.cc:299] Negotiation successful
0806 01:14:36.614063 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:14:36.615231 5350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:36.716075 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 5167
I20260806 01:14:36.740731 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 4120
I20260806 01:14:36.741818 4376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:36.867772 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 4120
I20260806 01:14:36.896976 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 4285
I20260806 01:14:36.898037 4676 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:37.018667 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 4285
2026-08-06T01:14:37Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10835 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:14:37 dist-test-slave-7nm7 krb5kdc[5578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:14:37 dist-test-slave-7nm7 krb5kdc[5578](info): set up 2 sockets
Aug 06 01:14:37 dist-test-slave-7nm7 krb5kdc[5578](info): commencing operation
krb5kdc: starting...
W20260806 01:14:38.385776 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:14:38 dist-test-slave-7nm7 krb5kdc[5578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:14:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:14:38Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.190.keytab.
Entry for principal oryx/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.190.keytab.
I20260806 01:14:38.682158 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:42737
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.190.keytab
--principal=oryx/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:42737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:39.027650 5594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:39.027928 5594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:39.028100 5594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:39.035784 5594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:39.035897 5594 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 01:14:39.036003 5594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:39.036095 5594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:39.036185 5594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:39.036293 5594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:14:39.047058 5594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:42737
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.190.keytab
--principal=oryx/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:42737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:39.048482 5594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:39.049947 5594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:39.061700 5605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:39.061968 5603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:39.061714 5602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:39.063022 5594 server_base.cc:1034] running on GCE node
I20260806 01:14:39.063855 5594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:39.065546 5594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:39.066811 5594 hybrid_clock.cc:648] HybridClock initialized: now 1785978879066754 us; error 56 us; skew 500 ppm
I20260806 01:14:39.067237 5594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:39 dist-test-slave-7nm7 krb5kdc[5578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978879, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:39.074769 5594 init.cc:377] Logged in from keytab as oryx/127.25.236.190@KRBTEST.COM (short username oryx)
I20260806 01:14:39.076318 5594 webserver.cc:460] Webserver started at http://127.25.236.190:37605/ using document root <none> and password file <none>
I20260806 01:14:39.076983 5594 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:39.077080 5594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:39.077478 5594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:39.080713 5594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "175df273e91c468eae0d35a24f131615"
format_stamp: "Formatted at 2026-08-06 01:14:39 on dist-test-slave-7nm7"
server_key: "dcbb2807937a4d300d8213808a981b8d"
server_key_iv: "70986b87f3a357d5431a2c026ee30252"
server_key_version: "encryptionkey@0"
I20260806 01:14:39.081470 5594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "175df273e91c468eae0d35a24f131615"
format_stamp: "Formatted at 2026-08-06 01:14:39 on dist-test-slave-7nm7"
server_key: "dcbb2807937a4d300d8213808a981b8d"
server_key_iv: "70986b87f3a357d5431a2c026ee30252"
server_key_version: "encryptionkey@0"
I20260806 01:14:39.086725 5594 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 01:14:39.090174 5617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:39.091552 5594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:14:39.091748 5594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "175df273e91c468eae0d35a24f131615"
format_stamp: "Formatted at 2026-08-06 01:14:39 on dist-test-slave-7nm7"
server_key: "dcbb2807937a4d300d8213808a981b8d"
server_key_iv: "70986b87f3a357d5431a2c026ee30252"
server_key_version: "encryptionkey@0"
I20260806 01:14:39.091975 5594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:39.103089 5594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:39.107344 5594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:39.107638 5594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:39.130940 5594 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:42737
I20260806 01:14:39.131047 5716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:42737 every 8 connection(s)
I20260806 01:14:39.132707 5594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:14:39.136691 5717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:39.141669 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 5594
I20260806 01:14:39.141894 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:14:39.142678 26546 external_mini_cluster.cc:1428] Setting key f691022db950671a27a839aaa0b231a7
I20260806 01:14:39.148689 5717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615: Bootstrap starting.
I20260806 01:14:39.152767 5717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:39.154048 5717 log.cc:824] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:39.157016 5717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615: No bootstrap required, opened a new log
Aug 06 01:14:39 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:39.168233 5722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:39.144710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49162 (local address 127.25.236.190:42737)
0806 01:14:39.145582 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:39.145593 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:39.145640 (+ 47us) server_negotiation.cc:407] Connection header received
0806 01:14:39.147411 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:39.147433 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:39.147796 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:39.148097 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:39.150117 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:39.151625 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:39.153487 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:39.154330 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:39.154472 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:39.158834 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:39.158845 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:39.158872 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:39.158931 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:39.163405 (+ 4474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:39.164400 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:39.164407 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:39.164413 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:39.164609 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:39.165095 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:39.165100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:39.165107 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:39.165718 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:39.165964 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:39.166657 (+ 693us) server_negotiation.cc:299] Negotiation successful
0806 01:14:39.167194 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":407,"threads_started":1}
I20260806 01:14:39.168355 5717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175df273e91c468eae0d35a24f131615" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 42737 } }
I20260806 01:14:39.168869 5717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:39.169003 5717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 175df273e91c468eae0d35a24f131615, State: Initialized, Role: FOLLOWER
I20260806 01:14:39.169699 5717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175df273e91c468eae0d35a24f131615" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 42737 } }
I20260806 01:14:39.170015 5717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:14:39.170205 5717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:14:39.170449 5717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:39.173333 5717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175df273e91c468eae0d35a24f131615" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 42737 } }
I20260806 01:14:39.173945 5717 leader_election.cc:302] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 175df273e91c468eae0d35a24f131615; no voters:
I20260806 01:14:39.174693 5717 leader_election.cc:288] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:14:39.174845 5728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:39.175642 5728 raft_consensus.cc:686] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [term 1 LEADER]: Becoming Leader. State: Replica: 175df273e91c468eae0d35a24f131615, State: Running, Role: LEADER
I20260806 01:14:39.176193 5728 consensus_queue.cc:237] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175df273e91c468eae0d35a24f131615" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 42737 } }
I20260806 01:14:39.176997 5717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:39.179500 5730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "175df273e91c468eae0d35a24f131615" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175df273e91c468eae0d35a24f131615" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 42737 } } }
I20260806 01:14:39.179984 5730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:39.181071 5731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 175df273e91c468eae0d35a24f131615. Latest consensus state: current_term: 1 leader_uuid: "175df273e91c468eae0d35a24f131615" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175df273e91c468eae0d35a24f131615" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 42737 } } }
I20260806 01:14:39.182331 5731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:39.183336 5741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:39.188923 5741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:39.198397 5741 catalog_manager.cc:1347] Generated new cluster ID: 65e9f7954e5d4fbfa53c4758d12bafd6
I20260806 01:14:39.198503 5741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:39.211865 5741 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:14:39.213759 5741 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:39.222493 5741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615: Generated new TSK 0
I20260806 01:14:39.223446 5741 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab.
Entry for principal oryx/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab.
I20260806 01:14:39.304350 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:42737
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab
--principal=oryx/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:39.673794 5758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:39.674089 5758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:39.674222 5758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:39.681957 5758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:39.682073 5758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:39.682158 5758 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:14:39.682287 5758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:14:39.692449 5758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab
--principal=oryx/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.236.190:42737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:39.693689 5758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:39.695068 5758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:39.709648 5767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:39.709688 5768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:39.709753 5770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:39.711180 5758 server_base.cc:1034] running on GCE node
I20260806 01:14:39.712373 5758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:39.713524 5758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:39.714808 5758 hybrid_clock.cc:648] HybridClock initialized: now 1785978879714747 us; error 58 us; skew 500 ppm
I20260806 01:14:39.715159 5758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:39 dist-test-slave-7nm7 krb5kdc[5578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978879, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:39.718426 5758 init.cc:377] Logged in from keytab as oryx/127.25.236.129@KRBTEST.COM (short username oryx)
I20260806 01:14:39.720134 5758 webserver.cc:460] Webserver started at http://127.25.236.129:38787/ using document root <none> and password file <none>
I20260806 01:14:39.720866 5758 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:39.720999 5758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:39.721349 5758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:39.724406 5758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "e254ab43ccf04cde8655ee5c07644be6"
format_stamp: "Formatted at 2026-08-06 01:14:39 on dist-test-slave-7nm7"
server_key: "1bffdd8413ac530fcac0e35e8851af9a"
server_key_iv: "26671fdca84a9a6c0d5f2fb9e00e4289"
server_key_version: "encryptionkey@0"
I20260806 01:14:39.725265 5758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "e254ab43ccf04cde8655ee5c07644be6"
format_stamp: "Formatted at 2026-08-06 01:14:39 on dist-test-slave-7nm7"
server_key: "1bffdd8413ac530fcac0e35e8851af9a"
server_key_iv: "26671fdca84a9a6c0d5f2fb9e00e4289"
server_key_version: "encryptionkey@0"
I20260806 01:14:39.730636 5758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:14:39.734473 5781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:39.735908 5758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:14:39.736111 5758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "e254ab43ccf04cde8655ee5c07644be6"
format_stamp: "Formatted at 2026-08-06 01:14:39 on dist-test-slave-7nm7"
server_key: "1bffdd8413ac530fcac0e35e8851af9a"
server_key_iv: "26671fdca84a9a6c0d5f2fb9e00e4289"
server_key_version: "encryptionkey@0"
I20260806 01:14:39.736363 5758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:39.766553 5758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:39.771821 5758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:39.772338 5758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:39.775400 5758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:39.775543 5758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:39.775719 5758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:39.775800 5758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:14:39 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978879, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:39.802928 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:39.776441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:42737 (local address 127.25.236.129:51093)
0806 01:14:39.777371 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:39.777427 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:39.779478 (+ 2051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:39.780280 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:39.780288 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:39.781229 (+ 941us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:39.782183 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:39.782208 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:39.784160 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:39.784167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:39.786675 (+ 2508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:39.786696 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:39.787854 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:39.787859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:39.788036 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:39.789523 (+ 1487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:39.789567 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:39.792745 (+ 3178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:39.796718 (+ 3973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:39.796735 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:39.796755 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:39.797530 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:39.797993 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:39.797998 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:39.798003 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:39.798287 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:39.798918 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:39.798932 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:39.799567 (+ 635us) client_negotiation.cc:769] Sending connection context
0806 01:14:39.799957 (+ 390us) client_negotiation.cc:240] Negotiation successful
0806 01:14:39.800705 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":406,"threads_started":1}
I20260806 01:14:39.803489 5794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:39.775845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:51093 (local address 127.25.236.190:42737)
0806 01:14:39.776359 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:39.776365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:39.777571 (+ 1206us) server_negotiation.cc:407] Connection header received
0806 01:14:39.779793 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:39.779799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:39.779905 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:39.780030 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:39.782497 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:39.783873 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:39.786878 (+ 3005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:39.787600 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:39.787768 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:39.793033 (+ 5265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:39.793039 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:39.793042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:39.793077 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:39.796540 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:39.797731 (+ 1191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:39.797737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:39.797741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:39.797868 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:39.798426 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:39.798431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:39.798434 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:39.798750 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:39.798938 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:39.802565 (+ 3627us) server_negotiation.cc:299] Negotiation successful
0806 01:14:39.802853 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":284,"threads_started":1}
I20260806 01:14:39.827898 5794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:39.811527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:51451 (local address 127.25.236.190:42737)
0806 01:14:39.811783 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:39.811792 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:39.812242 (+ 450us) server_negotiation.cc:407] Connection header received
0806 01:14:39.812440 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:39.812447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:39.812553 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:39.812676 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:39.813944 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:39.815282 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:39.817754 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:39.818322 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:39.818475 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:39.821180 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:39.821188 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:39.821195 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:39.821244 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:39.824917 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:39.825810 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:39.825816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:39.825820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:39.825947 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:39.826369 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:39.826374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:39.826378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:39.826660 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:39.826809 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:39.827283 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 01:14:39.827522 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 01:14:39.828023 5758 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:39933
I20260806 01:14:39.828087 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:39.811416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:42737 (local address 127.25.236.129:51451)
0806 01:14:39.812080 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:39.812124 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:39.812274 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:39.812730 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:39.812740 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:39.813426 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:39.813776 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:39.813787 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:39.815478 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:39.815486 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:39.817539 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:39.817562 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:39.818543 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:39.818550 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:39.818749 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:39.819518 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:39.819553 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:39.820880 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:39.825056 (+ 4176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:39.825062 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:39.825066 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:39.825686 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:39.826072 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:39.826077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:39.826081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:39.826222 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:39.826791 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:39.826795 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:39.827003 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 01:14:39.827169 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 01:14:39.827548 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":350,"threads_started":1}
I20260806 01:14:39.828616 5974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:39933 every 8 connection(s)
I20260806 01:14:39.829507 5758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:14:39.832118 5975 heartbeater.cc:347] Connected to a master server at 127.25.236.190:42737
I20260806 01:14:39.832403 5975 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:39.833127 5975 heartbeater.cc:511] Master 127.25.236.190:42737 requested a full tablet report, sending...
I20260806 01:14:39.835186 5643 ts_manager.cc:194] Registered new tserver with Master: e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933)
I20260806 01:14:39.837006 5643 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:51093
I20260806 01:14:39.838807 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 5758
I20260806 01:14:39.839011 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:14:39.839884 26546 external_mini_cluster.cc:1428] Setting key 31d5f7ae39867925e0eac974a27b85b0
WARNING: no policy specified for oryx/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab.
Entry for principal oryx/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab.
I20260806 01:14:39.899317 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:42737
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab
--principal=oryx/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:40.279047 5981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:40.279522 5981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:40.279736 5981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:40.288735 5981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:40.288877 5981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:40.289078 5981 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:14:40.289208 5981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:14:40.302734 5981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab
--principal=oryx/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.236.190:42737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:40.305123 5981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:40.307552 5981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:40.323467 5990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:40.323478 5991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:40.323772 5995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:40.324368 5981 server_base.cc:1034] running on GCE node
I20260806 01:14:40.325126 5981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:40.326146 5981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:40.327442 5981 hybrid_clock.cc:648] HybridClock initialized: now 1785978880327378 us; error 37 us; skew 500 ppm
I20260806 01:14:40.327855 5981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:40 dist-test-slave-7nm7 krb5kdc[5578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978880, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:40.332051 5981 init.cc:377] Logged in from keytab as oryx/127.25.236.130@KRBTEST.COM (short username oryx)
I20260806 01:14:40.334136 5981 webserver.cc:460] Webserver started at http://127.25.236.130:45343/ using document root <none> and password file <none>
I20260806 01:14:40.334901 5981 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:40.335044 5981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:40.335386 5981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:40.338757 5981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "1cab846ae46f44ec800e3312be75bca8"
format_stamp: "Formatted at 2026-08-06 01:14:40 on dist-test-slave-7nm7"
server_key: "6429f85ada3b95db799d3588df4f860d"
server_key_iv: "c64465eba0ae7540a0479c44ab62498d"
server_key_version: "encryptionkey@0"
I20260806 01:14:40.339655 5981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "1cab846ae46f44ec800e3312be75bca8"
format_stamp: "Formatted at 2026-08-06 01:14:40 on dist-test-slave-7nm7"
server_key: "6429f85ada3b95db799d3588df4f860d"
server_key_iv: "c64465eba0ae7540a0479c44ab62498d"
server_key_version: "encryptionkey@0"
I20260806 01:14:40.345383 5981 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:14:40.349220 6004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:40.350783 5981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:14:40.350979 5981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "1cab846ae46f44ec800e3312be75bca8"
format_stamp: "Formatted at 2026-08-06 01:14:40 on dist-test-slave-7nm7"
server_key: "6429f85ada3b95db799d3588df4f860d"
server_key_iv: "c64465eba0ae7540a0479c44ab62498d"
server_key_version: "encryptionkey@0"
I20260806 01:14:40.351246 5981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:40.374090 5981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:40.379070 5981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:40.379372 5981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:40.382689 5981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:40.382810 5981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:40.382968 5981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:40.383050 5981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:14:40 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978880, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:40.413479 6024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:40.383958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:50631 (local address 127.25.236.190:42737)
0806 01:14:40.384522 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:40.384526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:40.385968 (+ 1442us) server_negotiation.cc:407] Connection header received
0806 01:14:40.388135 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:40.388141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:40.388227 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:40.388357 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:40.391062 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:40.392918 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:40.396157 (+ 3239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:40.396944 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:40.397115 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:40.403217 (+ 6102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:40.403225 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:40.403232 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:40.403288 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:40.407799 (+ 4511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:40.409245 (+ 1446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:40.409254 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:40.409259 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:40.409434 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:40.410154 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:40.410161 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:40.410166 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:40.410689 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:40.410879 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:40.412656 (+ 1777us) server_negotiation.cc:299] Negotiation successful
0806 01:14:40.412953 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":352,"threads_started":1}
I20260806 01:14:40.416862 6026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:40.384644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:42737 (local address 127.25.236.130:50631)
0806 01:14:40.385787 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:40.385827 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:40.387857 (+ 2030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:40.388607 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:40.388615 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:40.389587 (+ 972us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:40.390812 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:40.390835 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:40.393468 (+ 2633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:40.393480 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:40.395928 (+ 2448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:40.395950 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:40.397251 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:40.397286 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:40.397665 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:40.398649 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:40.398698 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:40.402871 (+ 4173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:40.408041 (+ 5170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:40.408052 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:40.408081 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:40.409044 (+ 963us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:40.409621 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:40.409628 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:40.409634 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:40.409977 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:40.411002 (+ 1025us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:40.411012 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:40.411905 (+ 893us) client_negotiation.cc:769] Sending connection context
0806 01:14:40.413834 (+ 1929us) client_negotiation.cc:240] Negotiation successful
0806 01:14:40.414720 (+ 886us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"thread_start_us":569,"threads_started":1}
I20260806 01:14:40.430437 5981 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:35299
I20260806 01:14:40.430492 6192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:35299 every 8 connection(s)
I20260806 01:14:40.431912 5981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:14:40.434298 6194 heartbeater.cc:347] Connected to a master server at 127.25.236.190:42737
I20260806 01:14:40.434566 6194 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:40.435278 6194 heartbeater.cc:511] Master 127.25.236.190:42737 requested a full tablet report, sending...
I20260806 01:14:40.436777 5643 ts_manager.cc:194] Registered new tserver with Master: 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130:35299)
I20260806 01:14:40.437906 5643 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:50631
I20260806 01:14:40.439376 6172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:40.424075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:42737 (local address 127.25.236.130:50893)
0806 01:14:40.424809 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:40.424837 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:40.425013 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:40.425398 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:40.425402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:40.426066 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:40.426701 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:40.426714 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:40.428092 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:40.428098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:40.429580 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:40.429599 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:40.430612 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:40.430619 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:40.430819 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:40.431522 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:40.431542 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:40.432477 (+ 935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:40.436168 (+ 3691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:40.436173 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:40.436177 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:40.436691 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:40.437094 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:40.437097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:40.437100 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:40.437228 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:40.437887 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:40.437890 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:40.438102 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 01:14:40.438342 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 01:14:40.438710 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":376,"threads_started":1}
I20260806 01:14:40.440727 6024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:40.423916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:50893 (local address 127.25.236.190:42737)
0806 01:14:40.424136 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:40.424145 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:40.424938 (+ 793us) server_negotiation.cc:407] Connection header received
0806 01:14:40.425165 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:40.425177 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:40.425255 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:40.425361 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:40.426849 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:40.427946 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:40.429809 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:40.430406 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:40.430553 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:40.432718 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:40.432746 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:40.432752 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:40.432806 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:40.436000 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:40.436828 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:40.436833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:40.436836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:40.436972 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:40.437343 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:40.437348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:40.437352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:40.437771 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:40.437929 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:40.439929 (+ 2000us) server_negotiation.cc:299] Negotiation successful
0806 01:14:40.440237 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:14:40.443078 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 5981
I20260806 01:14:40.443270 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:14:40.443768 26546 external_mini_cluster.cc:1428] Setting key 4e03d270f011bff153b71fa2f565ac27
WARNING: no policy specified for oryx/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab.
Entry for principal oryx/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab.
I20260806 01:14:40.514730 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:42737
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab
--principal=oryx/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:14:40.840657 5975 heartbeater.cc:503] Master 127.25.236.190:42737 was elected leader, sending a full tablet report...
W20260806 01:14:40.898579 6198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:40.898901 6198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:40.899086 6198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:40.907239 6198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:40.907372 6198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:40.907552 6198 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:14:40.907701 6198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:14:40.918349 6198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab
--principal=oryx/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.236.190:42737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:40.919847 6198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:40.921435 6198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:40.935853 6210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:40.935822 6205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:40.936112 6204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:40.936889 6198 server_base.cc:1034] running on GCE node
I20260806 01:14:40.937489 6198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:40.938447 6198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:40.939689 6198 hybrid_clock.cc:648] HybridClock initialized: now 1785978880939607 us; error 45 us; skew 500 ppm
I20260806 01:14:40.940016 6198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:40 dist-test-slave-7nm7 krb5kdc[5578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978880, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:40.943604 6198 init.cc:377] Logged in from keytab as oryx/127.25.236.131@KRBTEST.COM (short username oryx)
I20260806 01:14:40.945434 6198 webserver.cc:460] Webserver started at http://127.25.236.131:44785/ using document root <none> and password file <none>
I20260806 01:14:40.946274 6198 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:40.946424 6198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:40.946741 6198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:40.949889 6198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "c6a00b734a74430280fe014db9169c52"
format_stamp: "Formatted at 2026-08-06 01:14:40 on dist-test-slave-7nm7"
server_key: "d236ed3d535b0a79da957f8adb6fb14e"
server_key_iv: "2ffb4d50a428d0477aa101a797709bbb"
server_key_version: "encryptionkey@0"
I20260806 01:14:40.950790 6198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "c6a00b734a74430280fe014db9169c52"
format_stamp: "Formatted at 2026-08-06 01:14:40 on dist-test-slave-7nm7"
server_key: "d236ed3d535b0a79da957f8adb6fb14e"
server_key_iv: "2ffb4d50a428d0477aa101a797709bbb"
server_key_version: "encryptionkey@0"
I20260806 01:14:40.956033 6198 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:14:40.960000 6221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:40.961372 6198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:14:40.961561 6198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "c6a00b734a74430280fe014db9169c52"
format_stamp: "Formatted at 2026-08-06 01:14:40 on dist-test-slave-7nm7"
server_key: "d236ed3d535b0a79da957f8adb6fb14e"
server_key_iv: "2ffb4d50a428d0477aa101a797709bbb"
server_key_version: "encryptionkey@0"
I20260806 01:14:40.961808 6198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:40.991954 6198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:40.996683 6198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:40.996973 6198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:40.999754 6198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:40.999867 6198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:41.000029 6198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:41.000109 6198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:14:41 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978880, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:41.028860 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.000574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:45175 (local address 127.25.236.190:42737)
0806 01:14:41.005672 (+ 5098us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.005677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.005695 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:14:41.005754 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.005758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.005819 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.005950 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:41.008377 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.009647 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.013074 (+ 3427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.013817 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.013979 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.020064 (+ 6085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:41.020085 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:41.020093 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:41.020151 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:41.024323 (+ 4172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:41.025435 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:41.025443 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:41.025449 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:41.025638 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:41.026287 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:41.026295 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:41.026301 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:41.026725 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:41.026909 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.028051 (+ 1142us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.028349 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4976,"thread_start_us":4831,"threads_started":1}
I20260806 01:14:41.030500 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.001117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:42737 (local address 127.25.236.131:45175)
0806 01:14:41.001991 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.002037 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.003883 (+ 1846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.006319 (+ 2436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.006333 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.007248 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:41.008139 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.008173 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.009878 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.009887 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.012880 (+ 2993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.012901 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.014139 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.014147 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.014645 (+ 498us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.015558 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:41.015601 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:41.019740 (+ 4139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:41.024460 (+ 4720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:41.024467 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:41.024498 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:41.025273 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:41.025771 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:41.025779 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:41.025784 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:41.026093 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:41.026889 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:41.026902 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:41.027491 (+ 589us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.027892 (+ 401us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.028599 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":398,"threads_started":1}
I20260806 01:14:41.048085 6198 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:38439
I20260806 01:14:41.048133 6403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:38439 every 8 connection(s)
I20260806 01:14:41.049470 6198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:14:41.050251 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 6198
I20260806 01:14:41.050501 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:14:41.051235 26546 external_mini_cluster.cc:1428] Setting key f81cc71779712053f0bf55a0f1459b64
I20260806 01:14:41.052031 6404 heartbeater.cc:347] Connected to a master server at 127.25.236.190:42737
I20260806 01:14:41.052309 6404 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:41.053273 6404 heartbeater.cc:511] Master 127.25.236.190:42737 requested a full tablet report, sending...
I20260806 01:14:41.055019 5642 ts_manager.cc:194] Registered new tserver with Master: c6a00b734a74430280fe014db9169c52 (127.25.236.131:38439)
I20260806 01:14:41.056344 5642 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:45175
I20260806 01:14:41.057288 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.038389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:54451 (local address 127.25.236.190:42737)
0806 01:14:41.038661 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.038669 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.043047 (+ 4378us) server_negotiation.cc:407] Connection header received
0806 01:14:41.043300 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.043305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.043400 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.043528 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:41.044935 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.045864 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.047525 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.047966 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.048105 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.050277 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:41.050285 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:41.050293 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:41.050338 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:41.054097 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:41.055050 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:41.055055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:41.055059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:41.055225 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:41.055689 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:41.055694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:41.055698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:41.056091 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:41.056268 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.056728 (+ 460us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.057009 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 01:14:41.057502 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.038575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:42737 (local address 127.25.236.131:54451)
0806 01:14:41.042915 (+ 4340us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.042938 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.043135 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.043747 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.043752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.044389 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:41.044770 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.044784 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.046015 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.046020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.047365 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.047378 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.048201 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.048209 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.048420 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.049104 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:41.049122 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:41.049965 (+ 843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:41.054278 (+ 4313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:41.054283 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:41.054287 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:41.054882 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:41.055366 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:41.055371 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:41.055374 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:41.055550 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:41.056264 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:41.056269 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:41.056448 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.056568 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.057005 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4156,"thread_start_us":3829,"threads_started":1}
I20260806 01:14:41.067845 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:14:41 dist-test-slave-7nm7 krb5kdc[5578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978881, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:14:41 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:41 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:14:41.129608 6231 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:41.118029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49170 (local address 127.25.236.190:42737)
0806 01:14:41.118327 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.118335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.118549 (+ 214us) server_negotiation.cc:407] Connection header received
0806 01:14:41.118807 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.118812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.118902 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.119042 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:41.120625 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.121363 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.122829 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.123245 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.123396 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.129322 (+ 5926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49170: BlockingRecv error: recv got EOF from 127.0.0.1:49170 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 06 01:14:41 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978881, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:41.150166 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.132959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49182 (local address 127.25.236.190:42737)
0806 01:14:41.133145 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.133151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.135690 (+ 2539us) server_negotiation.cc:407] Connection header received
0806 01:14:41.135835 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.135843 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.135927 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.136041 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:41.137177 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.138300 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.139522 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.139934 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.140387 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.144481 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:41.144489 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:41.144493 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:41.144545 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:41.147385 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:41.148044 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:41.148048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:41.148050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:41.148192 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:41.148630 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:41.148634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:41.148637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:41.148965 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:41.149206 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.149585 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.149826 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 01:14:41.154616 5642 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:14:41.162194 5642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:14:41.201001 6431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.175442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:35299 (local address 127.0.0.1:43124)
0806 01:14:41.188929 (+ 13487us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.188992 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.189270 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.191285 (+ 2015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.191298 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.191361 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:41.191988 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.192009 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.193852 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.193863 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.197053 (+ 3190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.197069 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.199702 (+ 2633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.199709 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.199904 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.199983 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.200279 (+ 296us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.200518 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13101,"mutex_wait_us":1,"thread_start_us":16659,"threads_started":1}
I20260806 01:14:41.201911 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.175622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43124 (local address 127.25.236.130:35299)
0806 01:14:41.189999 (+ 14377us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.190011 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.190048 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:14:41.190187 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.190198 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.190517 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.190746 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:41.192248 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.193656 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.197621 (+ 3965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.199474 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.199681 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.199870 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.201398 (+ 1528us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.201600 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14165,"thread_start_us":15328,"threads_started":1}
I20260806 01:14:41.202847 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.176182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:38439 (local address 127.0.0.1:56178)
0806 01:14:41.192426 (+ 16244us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.192448 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.192673 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.193487 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.193491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.193515 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:41.193945 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.193956 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.195675 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.195681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.199171 (+ 3490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.199188 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.202155 (+ 2967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.202161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.202387 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.202411 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.202490 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.202589 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16098,"mutex_wait_us":259,"thread_start_us":15055,"threads_started":1}
I20260806 01:14:41.203569 6435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.176520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56178 (local address 127.25.236.131:38439)
0806 01:14:41.190982 (+ 14462us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.190994 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.192557 (+ 1563us) server_negotiation.cc:407] Connection header received
0806 01:14:41.192907 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.192916 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.193140 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.193335 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:41.194207 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.195519 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.199614 (+ 4095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.201893 (+ 2279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.202162 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.202427 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.202548 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.202883 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14208,"thread_start_us":13915,"threads_started":1}
I20260806 01:14:41.206028 6434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.175629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:39933 (local address 127.0.0.1:54890)
0806 01:14:41.191591 (+ 15962us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.191611 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.191800 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.192746 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.192751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.192777 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:41.193301 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.193316 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.195758 (+ 2442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.195765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.198714 (+ 2949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.198729 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.205221 (+ 6492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.205228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.205403 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.205434 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.205528 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.205670 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15800,"mutex_wait_us":148,"thread_start_us":12749,"threads_started":1}
I20260806 01:14:41.206696 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.176693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54890 (local address 127.25.236.129:39933)
0806 01:14:41.186616 (+ 9923us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.186630 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.191760 (+ 5130us) server_negotiation.cc:407] Connection header received
0806 01:14:41.192014 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.192025 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.192278 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.192507 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:41.193534 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.195227 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.198980 (+ 3753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.201915 (+ 2935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.204862 (+ 2947us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.205056 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.206319 (+ 1263us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.206475 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9666,"thread_start_us":9519,"threads_started":1}
I20260806 01:14:41.206933 6088 tablet_service.cc:1467] Processing CreateTablet for tablet 538c971afefb4b4bb2b1cab57d4bc8ad (DEFAULT_TABLE table=test-table [id=8e6b05452b7c4800a82cf1c07f0b7e40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:41.207182 6295 tablet_service.cc:1467] Processing CreateTablet for tablet 538c971afefb4b4bb2b1cab57d4bc8ad (DEFAULT_TABLE table=test-table [id=8e6b05452b7c4800a82cf1c07f0b7e40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:41.208582 6295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538c971afefb4b4bb2b1cab57d4bc8ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:41.208582 6088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538c971afefb4b4bb2b1cab57d4bc8ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:41.209444 5864 tablet_service.cc:1467] Processing CreateTablet for tablet 538c971afefb4b4bb2b1cab57d4bc8ad (DEFAULT_TABLE table=test-table [id=8e6b05452b7c4800a82cf1c07f0b7e40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:41.210927 5864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538c971afefb4b4bb2b1cab57d4bc8ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:41.226090 6439 tablet_bootstrap.cc:492] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6: Bootstrap starting.
I20260806 01:14:41.227326 6440 tablet_bootstrap.cc:492] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8: Bootstrap starting.
I20260806 01:14:41.228287 6441 tablet_bootstrap.cc:492] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52: Bootstrap starting.
I20260806 01:14:41.231410 6440 tablet_bootstrap.cc:654] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:41.231432 6439 tablet_bootstrap.cc:654] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:41.231516 6441 tablet_bootstrap.cc:654] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:41.232707 6441 log.cc:824] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:41.232759 6439 log.cc:824] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:41.232753 6440 log.cc:824] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:41.242159 6439 tablet_bootstrap.cc:492] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6: No bootstrap required, opened a new log
I20260806 01:14:41.242656 6439 ts_tablet_manager.cc:1397] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.003s
I20260806 01:14:41.246634 6441 tablet_bootstrap.cc:492] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52: No bootstrap required, opened a new log
I20260806 01:14:41.246958 6441 ts_tablet_manager.cc:1397] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.011s
I20260806 01:14:41.247594 6440 tablet_bootstrap.cc:492] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8: No bootstrap required, opened a new log
I20260806 01:14:41.247879 6440 ts_tablet_manager.cc:1397] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.016s
I20260806 01:14:41.254565 6439 raft_consensus.cc:348] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.255060 6439 raft_consensus.cc:374] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:41.255168 6439 raft_consensus.cc:729] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e254ab43ccf04cde8655ee5c07644be6, State: Initialized, Role: FOLLOWER
I20260806 01:14:41.255429 6441 raft_consensus.cc:348] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.255951 6441 raft_consensus.cc:374] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:41.255851 6439 consensus_queue.cc:260] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.256090 6441 raft_consensus.cc:729] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6a00b734a74430280fe014db9169c52, State: Initialized, Role: FOLLOWER
I20260806 01:14:41.256696 6441 consensus_queue.cc:260] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.257967 6404 heartbeater.cc:503] Master 127.25.236.190:42737 was elected leader, sending a full tablet report...
I20260806 01:14:41.258834 6441 ts_tablet_manager.cc:1428] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52: Time spent starting tablet: real 0.012s user 0.010s sys 0.002s
I20260806 01:14:41.258862 6440 raft_consensus.cc:348] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.259243 6440 raft_consensus.cc:374] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:41.259353 6440 raft_consensus.cc:729] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cab846ae46f44ec800e3312be75bca8, State: Initialized, Role: FOLLOWER
I20260806 01:14:41.260185 6440 consensus_queue.cc:260] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.261730 6194 heartbeater.cc:503] Master 127.25.236.190:42737 was elected leader, sending a full tablet report...
I20260806 01:14:41.263324 6440 ts_tablet_manager.cc:1428] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
I20260806 01:14:41.266166 6439 ts_tablet_manager.cc:1428] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6: Time spent starting tablet: real 0.023s user 0.019s sys 0.001s
I20260806 01:14:41.289436 6456 raft_consensus.cc:482] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:14:41.289773 6456 raft_consensus.cc:504] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.291539 6456 leader_election.cc:288] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130:35299), c6a00b734a74430280fe014db9169c52 (127.25.236.131:38439)
I20260806 01:14:41.300805 6435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.292020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:56451 (local address 127.25.236.131:38439)
0806 01:14:41.292298 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.292305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.292872 (+ 567us) server_negotiation.cc:407] Connection header received
0806 01:14:41.293124 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.293133 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.293203 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.293307 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:41.293858 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.294787 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.297640 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.299627 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.299828 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.299909 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.300363 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.300527 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260806 01:14:41.300895 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.291910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:38439 (local address 127.25.236.129:56451)
0806 01:14:41.292754 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.292770 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.292967 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.293327 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.293331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.293359 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:41.293715 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.293724 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.294929 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.294936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.297277 (+ 2341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.297321 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.299897 (+ 2576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.299902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.300132 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.300183 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.300315 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.300464 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":985,"threads_started":1}
I20260806 01:14:41.300948 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.292107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:50665 (local address 127.25.236.130:35299)
0806 01:14:41.292298 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.292305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.293410 (+ 1105us) server_negotiation.cc:407] Connection header received
0806 01:14:41.293684 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.293690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.293756 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.293865 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:41.294495 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.295231 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.298102 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.299902 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.300056 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.300127 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.300530 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.300688 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:14:41.301141 6459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.292004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:35299 (local address 127.25.236.129:50665)
0806 01:14:41.293281 (+ 1277us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.293299 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.293516 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.293889 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.293893 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.293920 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:41.294332 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.294343 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.295457 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.295463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.297771 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.297787 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.300145 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.300151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.300352 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.300382 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.300466 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.300579 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1068,"thread_start_us":444,"threads_started":1}
I20260806 01:14:41.302052 6117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538c971afefb4b4bb2b1cab57d4bc8ad" candidate_uuid: "e254ab43ccf04cde8655ee5c07644be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cab846ae46f44ec800e3312be75bca8" is_pre_election: true
I20260806 01:14:41.301952 6329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538c971afefb4b4bb2b1cab57d4bc8ad" candidate_uuid: "e254ab43ccf04cde8655ee5c07644be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6a00b734a74430280fe014db9169c52" is_pre_election: true
I20260806 01:14:41.302667 6117 raft_consensus.cc:2454] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e254ab43ccf04cde8655ee5c07644be6 in term 0.
I20260806 01:14:41.303035 6329 raft_consensus.cc:2454] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e254ab43ccf04cde8655ee5c07644be6 in term 0.
I20260806 01:14:41.303437 5785 leader_election.cc:302] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cab846ae46f44ec800e3312be75bca8, e254ab43ccf04cde8655ee5c07644be6; no voters:
I20260806 01:14:41.303922 6456 raft_consensus.cc:2781] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:14:41.304106 6456 raft_consensus.cc:482] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:14:41.304241 6456 raft_consensus.cc:3037] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Advancing to term 1
W20260806 01:14:41.306322 6406 tablet.cc:2369] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:14:41.306867 6456 raft_consensus.cc:504] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.307515 6456 leader_election.cc:288] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [CANDIDATE]: Term 1 election: Requested vote from peers 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130:35299), c6a00b734a74430280fe014db9169c52 (127.25.236.131:38439)
I20260806 01:14:41.308111 6117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538c971afefb4b4bb2b1cab57d4bc8ad" candidate_uuid: "e254ab43ccf04cde8655ee5c07644be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cab846ae46f44ec800e3312be75bca8"
I20260806 01:14:41.308109 6329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538c971afefb4b4bb2b1cab57d4bc8ad" candidate_uuid: "e254ab43ccf04cde8655ee5c07644be6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6a00b734a74430280fe014db9169c52"
I20260806 01:14:41.308348 6329 raft_consensus.cc:3037] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:41.308342 6117 raft_consensus.cc:3037] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:41.311834 6117 raft_consensus.cc:2454] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e254ab43ccf04cde8655ee5c07644be6 in term 1.
I20260806 01:14:41.311834 6329 raft_consensus.cc:2454] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e254ab43ccf04cde8655ee5c07644be6 in term 1.
I20260806 01:14:41.312346 5786 leader_election.cc:302] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6a00b734a74430280fe014db9169c52, e254ab43ccf04cde8655ee5c07644be6; no voters:
I20260806 01:14:41.312697 6456 raft_consensus.cc:2781] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:41.313190 6456 raft_consensus.cc:686] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [term 1 LEADER]: Becoming Leader. State: Replica: e254ab43ccf04cde8655ee5c07644be6, State: Running, Role: LEADER
I20260806 01:14:41.313800 6456 consensus_queue.cc:237] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.320050 5642 catalog_manager.cc:5495] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 reported cstate change: term changed from 0 to 1, leader changed from <none> to e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129). New cstate: current_term: 1 leader_uuid: "e254ab43ccf04cde8655ee5c07644be6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } health_report { overall_health: UNKNOWN } } }
W20260806 01:14:41.333186 5977 tablet.cc:2369] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:14:41.365000 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.359039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54906 (local address 127.25.236.129:39933)
0806 01:14:41.359282 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.359288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.359308 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:14:41.359516 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.359520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.359622 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.359786 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:14:41.360436 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.361199 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.362681 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.363211 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.363353 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.363841 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:14:41.364377 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:14:41.364535 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.364626 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.364707 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:14:41.373282 6117 raft_consensus.cc:1261] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Refusing update from remote peer e254ab43ccf04cde8655ee5c07644be6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:41.373274 6329 raft_consensus.cc:1261] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Refusing update from remote peer e254ab43ccf04cde8655ee5c07644be6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:41.374326 6461 consensus_queue.cc:1035] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:41.374742 6456 consensus_queue.cc:1035] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:41.404767 5642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:41.405014 5642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:41.409197 5642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615: Sending DeleteTablet for 3 replicas of tablet 538c971afefb4b4bb2b1cab57d4bc8ad
I20260806 01:14:41.410482 5864 tablet_service.cc:1514] Processing DeleteTablet for tablet 538c971afefb4b4bb2b1cab57d4bc8ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:41 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:54890
I20260806 01:14:41.410543 6295 tablet_service.cc:1514] Processing DeleteTablet for tablet 538c971afefb4b4bb2b1cab57d4bc8ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:41 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:56178
I20260806 01:14:41.410535 6088 tablet_service.cc:1514] Processing DeleteTablet for tablet 538c971afefb4b4bb2b1cab57d4bc8ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:41 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:43124
I20260806 01:14:41.414871 5643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:14:41.415918 5643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:14:41.427273 6086 tablet_service.cc:1467] Processing CreateTablet for tablet 265514c867c5477ba1d17a60e29e55c4 (DEFAULT_TABLE table=test-table [id=88550a24f17048f6bfdb29513f4d00f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:41.427976 6086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265514c867c5477ba1d17a60e29e55c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:41.427910 5862 tablet_service.cc:1467] Processing CreateTablet for tablet 265514c867c5477ba1d17a60e29e55c4 (DEFAULT_TABLE table=test-table [id=88550a24f17048f6bfdb29513f4d00f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:41.428447 5862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265514c867c5477ba1d17a60e29e55c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:41.429026 6294 tablet_service.cc:1467] Processing CreateTablet for tablet 265514c867c5477ba1d17a60e29e55c4 (DEFAULT_TABLE table=test-table [id=88550a24f17048f6bfdb29513f4d00f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:41.429857 6294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265514c867c5477ba1d17a60e29e55c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:41.435933 6439 tablet_bootstrap.cc:492] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6: Bootstrap starting.
I20260806 01:14:41.437674 6439 tablet_bootstrap.cc:654] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:41.438138 6441 tablet_bootstrap.cc:492] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52: Bootstrap starting.
I20260806 01:14:41.438581 6478 tablet_replica.cc:330] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52: stopping tablet replica
I20260806 01:14:41.438689 6440 tablet_bootstrap.cc:492] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8: Bootstrap starting.
I20260806 01:14:41.439211 6478 raft_consensus.cc:2229] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:41.439841 6478 raft_consensus.cc:2258] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:41.440502 6441 tablet_bootstrap.cc:654] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:41.441581 6440 tablet_bootstrap.cc:654] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:41.452137 6478 ts_tablet_manager.cc:1905] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:41.455561 6441 tablet_bootstrap.cc:492] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52: No bootstrap required, opened a new log
I20260806 01:14:41.457805 6441 ts_tablet_manager.cc:1397] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.009s
I20260806 01:14:41.458017 6482 tablet_replica.cc:330] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8: stopping tablet replica
I20260806 01:14:41.458652 6482 raft_consensus.cc:2229] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:41.459100 6479 tablet_replica.cc:330] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6: stopping tablet replica
I20260806 01:14:41.459178 6441 raft_consensus.cc:348] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.459440 6441 raft_consensus.cc:374] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:41.459535 6441 raft_consensus.cc:729] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6a00b734a74430280fe014db9169c52, State: Initialized, Role: FOLLOWER
I20260806 01:14:41.459813 6441 consensus_queue.cc:260] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.460107 6482 raft_consensus.cc:2258] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:41.460263 6479 raft_consensus.cc:2229] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:14:41.461464 6479 raft_consensus.cc:2258] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:41.463660 6441 ts_tablet_manager.cc:1428] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260806 01:14:41.466518 6479 ts_tablet_manager.cc:1905] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:41.470278 6440 tablet_bootstrap.cc:492] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8: No bootstrap required, opened a new log
I20260806 01:14:41.470530 6440 ts_tablet_manager.cc:1397] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.001s
I20260806 01:14:41.471691 6440 raft_consensus.cc:348] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.472021 6440 raft_consensus.cc:374] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:41.472136 6440 raft_consensus.cc:729] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cab846ae46f44ec800e3312be75bca8, State: Initialized, Role: FOLLOWER
I20260806 01:14:41.472486 6440 consensus_queue.cc:260] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.473284 6440 ts_tablet_manager.cc:1428] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260806 01:14:41.475194 6439 tablet_bootstrap.cc:492] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6: No bootstrap required, opened a new log
I20260806 01:14:41.475600 6439 ts_tablet_manager.cc:1397] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6: Time spent bootstrapping tablet: real 0.040s user 0.016s sys 0.005s
I20260806 01:14:41.476625 6439 raft_consensus.cc:348] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.476958 6439 raft_consensus.cc:374] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:41.477064 6439 raft_consensus.cc:729] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e254ab43ccf04cde8655ee5c07644be6, State: Initialized, Role: FOLLOWER
I20260806 01:14:41.477680 6439 consensus_queue.cc:260] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.478623 6439 ts_tablet_manager.cc:1428] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 01:14:41.480137 6478 ts_tablet_manager.cc:1918] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:41.480391 6478 log.cc:1196] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/538c971afefb4b4bb2b1cab57d4bc8ad
I20260806 01:14:41.480979 6478 ts_tablet_manager.cc:1939] T 538c971afefb4b4bb2b1cab57d4bc8ad P c6a00b734a74430280fe014db9169c52: Deleting consensus metadata
I20260806 01:14:41.482018 6479 ts_tablet_manager.cc:1918] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:41.482197 6479 log.cc:1196] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/538c971afefb4b4bb2b1cab57d4bc8ad
I20260806 01:14:41.482874 6479 ts_tablet_manager.cc:1939] T 538c971afefb4b4bb2b1cab57d4bc8ad P e254ab43ccf04cde8655ee5c07644be6: Deleting consensus metadata
I20260806 01:14:41.483403 5621 catalog_manager.cc:4841] TS c6a00b734a74430280fe014db9169c52 (127.25.236.131:38439): tablet 538c971afefb4b4bb2b1cab57d4bc8ad (table test-table [id=8e6b05452b7c4800a82cf1c07f0b7e40]) successfully deleted
I20260806 01:14:41.484786 5618 catalog_manager.cc:4841] TS e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933): tablet 538c971afefb4b4bb2b1cab57d4bc8ad (table test-table [id=8e6b05452b7c4800a82cf1c07f0b7e40]) successfully deleted
I20260806 01:14:41.485893 6482 ts_tablet_manager.cc:1905] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:41.493368 6482 ts_tablet_manager.cc:1918] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:41.493568 6482 log.cc:1196] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/538c971afefb4b4bb2b1cab57d4bc8ad
I20260806 01:14:41.494170 6482 ts_tablet_manager.cc:1939] T 538c971afefb4b4bb2b1cab57d4bc8ad P 1cab846ae46f44ec800e3312be75bca8: Deleting consensus metadata
I20260806 01:14:41.496188 5619 catalog_manager.cc:4841] TS 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130:35299): tablet 538c971afefb4b4bb2b1cab57d4bc8ad (table test-table [id=8e6b05452b7c4800a82cf1c07f0b7e40]) successfully deleted
I20260806 01:14:41.609470 6458 raft_consensus.cc:482] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:14:41.609799 6458 raft_consensus.cc:504] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.611629 6458 leader_election.cc:288] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933), c6a00b734a74430280fe014db9169c52 (127.25.236.131:38439)
I20260806 01:14:41.620741 6435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.612347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:36175 (local address 127.25.236.131:38439)
0806 01:14:41.612545 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.612552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.613679 (+ 1127us) server_negotiation.cc:407] Connection header received
0806 01:14:41.613855 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.613862 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.613945 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.614054 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:41.615028 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.615931 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.618504 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.619685 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.619878 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.619966 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.620318 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.620454 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:14:41.620878 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.611952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:38439 (local address 127.25.236.130:36175)
0806 01:14:41.613550 (+ 1598us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.613569 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.613714 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.614092 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.614097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.614125 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:41.614587 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.614600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.616083 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.616092 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.618292 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.618309 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.619905 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.619912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.620123 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.620150 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.620281 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.620421 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1473,"mutex_wait_us":4,"thread_start_us":937,"threads_started":1}
I20260806 01:14:41.621630 6329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265514c867c5477ba1d17a60e29e55c4" candidate_uuid: "1cab846ae46f44ec800e3312be75bca8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6a00b734a74430280fe014db9169c52" is_pre_election: true
I20260806 01:14:41.621912 6329 raft_consensus.cc:2454] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cab846ae46f44ec800e3312be75bca8 in term 0.
I20260806 01:14:41.622506 6007 leader_election.cc:302] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cab846ae46f44ec800e3312be75bca8, c6a00b734a74430280fe014db9169c52; no voters:
I20260806 01:14:41.622956 6458 raft_consensus.cc:2781] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:14:41.623127 6458 raft_consensus.cc:482] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:14:41.623260 6458 raft_consensus.cc:3037] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:41.623498 6493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.611932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:39933 (local address 127.25.236.130:59629)
0806 01:14:41.613325 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.613349 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.613605 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.614678 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.614682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.614712 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:41.615163 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.615177 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.616727 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.616733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.619003 (+ 2270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.619018 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.622459 (+ 3441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.622465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.622658 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.622693 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.622835 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.622977 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1220,"mutex_wait_us":29,"thread_start_us":1302,"threads_started":1}
I20260806 01:14:41.625700 6458 raft_consensus.cc:504] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.626588 6458 leader_election.cc:288] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [CANDIDATE]: Term 1 election: Requested vote from peers e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933), c6a00b734a74430280fe014db9169c52 (127.25.236.131:38439)
I20260806 01:14:41.627198 6329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265514c867c5477ba1d17a60e29e55c4" candidate_uuid: "1cab846ae46f44ec800e3312be75bca8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6a00b734a74430280fe014db9169c52"
I20260806 01:14:41.627391 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.612371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:59629 (local address 127.25.236.129:39933)
0806 01:14:41.612656 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.612667 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.613465 (+ 798us) server_negotiation.cc:407] Connection header received
0806 01:14:41.613855 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.613862 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.613945 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.614039 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:41.615394 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.616566 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.619196 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.621395 (+ 2199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.622393 (+ 998us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.622477 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.626798 (+ 4321us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.626978 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 01:14:41.627449 6329 raft_consensus.cc:3037] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:41.628391 5894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265514c867c5477ba1d17a60e29e55c4" candidate_uuid: "1cab846ae46f44ec800e3312be75bca8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e254ab43ccf04cde8655ee5c07644be6" is_pre_election: true
I20260806 01:14:41.628506 5893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265514c867c5477ba1d17a60e29e55c4" candidate_uuid: "1cab846ae46f44ec800e3312be75bca8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e254ab43ccf04cde8655ee5c07644be6"
I20260806 01:14:41.628921 5894 raft_consensus.cc:2454] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cab846ae46f44ec800e3312be75bca8 in term 0.
I20260806 01:14:41.630575 6329 raft_consensus.cc:2454] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cab846ae46f44ec800e3312be75bca8 in term 1.
I20260806 01:14:41.631038 6007 leader_election.cc:302] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cab846ae46f44ec800e3312be75bca8, c6a00b734a74430280fe014db9169c52; no voters: e254ab43ccf04cde8655ee5c07644be6
I20260806 01:14:41.631415 6458 raft_consensus.cc:2781] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:41.631992 6458 raft_consensus.cc:686] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [term 1 LEADER]: Becoming Leader. State: Replica: 1cab846ae46f44ec800e3312be75bca8, State: Running, Role: LEADER
I20260806 01:14:41.632699 6458 consensus_queue.cc:237] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.637048 5643 catalog_manager.cc:5495] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130). New cstate: current_term: 1 leader_uuid: "1cab846ae46f44ec800e3312be75bca8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } health_report { overall_health: UNKNOWN } } }
W20260806 01:14:41.685959 6195 tablet.cc:2369] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 01:14:41 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978879, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.190@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:41.734714 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.716669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:42737 (local address 127.0.0.1:49184)
0806 01:14:41.717416 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.717447 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.717739 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.718686 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.718691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.719839 (+ 1148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:41.720066 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.720076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.721507 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.721516 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.722599 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.722611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.723374 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.723380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.723561 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.724568 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:41.724594 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:41.727286 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:41.731557 (+ 4271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:41.731571 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:41.731578 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:41.732048 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:41.732563 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:41.732570 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:41.732575 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:41.732751 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:41.733395 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:41.733404 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:41.733579 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.733791 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.734145 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":306,"threads_started":1}
I20260806 01:14:41.734769 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.716943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.25.236.190:42737)
0806 01:14:41.718329 (+ 1386us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.718336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.718357 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:14:41.718414 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.718420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.718501 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.718618 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:41.720404 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.721362 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.722746 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.723183 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.723344 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.727559 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:41.727565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:41.727569 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:41.727601 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:41.731309 (+ 3708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:41.732203 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:41.732209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:41.732214 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:41.732395 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:41.732878 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:41.732886 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:41.732890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:41.733206 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:41.733425 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.733895 (+ 470us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.734307 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":172,"server-negotiator.queue_time_us":1182,"thread_start_us":984,"threads_started":1}
I20260806 01:14:41.742784 5643 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:49184:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 01:14:41.744694 5643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:14:41.753249 5864 tablet_service.cc:1467] Processing CreateTablet for tablet 2e737dc1b2d64fa2a01e16de302e1fc2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5010217007ef41128ab98eef1c8cf757]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:14:41.753687 6294 tablet_service.cc:1467] Processing CreateTablet for tablet 2e737dc1b2d64fa2a01e16de302e1fc2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5010217007ef41128ab98eef1c8cf757]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:14:41.753908 5864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e737dc1b2d64fa2a01e16de302e1fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:41.754180 6294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e737dc1b2d64fa2a01e16de302e1fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:41.754782 6088 tablet_service.cc:1467] Processing CreateTablet for tablet 2e737dc1b2d64fa2a01e16de302e1fc2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5010217007ef41128ab98eef1c8cf757]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:14:41.755370 6088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e737dc1b2d64fa2a01e16de302e1fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:41.761858 6439 tablet_bootstrap.cc:492] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6: Bootstrap starting.
I20260806 01:14:41.763715 6440 tablet_bootstrap.cc:492] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8: Bootstrap starting.
I20260806 01:14:41.763867 6439 tablet_bootstrap.cc:654] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:41.765075 6440 tablet_bootstrap.cc:654] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:41.766399 6441 tablet_bootstrap.cc:492] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52: Bootstrap starting.
I20260806 01:14:41.767928 6441 tablet_bootstrap.cc:654] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:41.770279 6440 tablet_bootstrap.cc:492] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8: No bootstrap required, opened a new log
I20260806 01:14:41.770573 6440 ts_tablet_manager.cc:1397] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260806 01:14:41.771287 6441 tablet_bootstrap.cc:492] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52: No bootstrap required, opened a new log
I20260806 01:14:41.771472 6441 ts_tablet_manager.cc:1397] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 01:14:41.771623 6440 raft_consensus.cc:348] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.771991 6440 raft_consensus.cc:374] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:41.772131 6440 raft_consensus.cc:729] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cab846ae46f44ec800e3312be75bca8, State: Initialized, Role: FOLLOWER
I20260806 01:14:41.772377 6441 raft_consensus.cc:348] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.772650 6441 raft_consensus.cc:374] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:41.772742 6441 raft_consensus.cc:729] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6a00b734a74430280fe014db9169c52, State: Initialized, Role: FOLLOWER
I20260806 01:14:41.773002 6441 consensus_queue.cc:260] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.772536 6440 consensus_queue.cc:260] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.773670 6441 ts_tablet_manager.cc:1428] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 01:14:41.773533 6457 tablet_replica.cc:439] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } } }
I20260806 01:14:41.773906 6457 tablet_replica.cc:442] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:14:41.773856 6458 tablet_replica.cc:439] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } } }
I20260806 01:14:41.774297 6458 tablet_replica.cc:442] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:14:41.775709 6440 ts_tablet_manager.cc:1428] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260806 01:14:41.776113 6457 raft_consensus.cc:482] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:14:41.776297 6457 raft_consensus.cc:504] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.778110 6457 leader_election.cc:288] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933), 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130:35299)
I20260806 01:14:41.783205 6439 tablet_bootstrap.cc:492] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6: No bootstrap required, opened a new log
I20260806 01:14:41.783417 6439 ts_tablet_manager.cc:1397] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.009s
I20260806 01:14:41.785394 6439 raft_consensus.cc:348] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.785681 6439 raft_consensus.cc:374] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:41.785792 6439 raft_consensus.cc:729] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e254ab43ccf04cde8655ee5c07644be6, State: Initialized, Role: FOLLOWER
I20260806 01:14:41.786074 6439 consensus_queue.cc:260] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.786577 6456 tablet_replica.cc:439] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } } }
I20260806 01:14:41.786870 6456 tablet_replica.cc:442] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:14:41.787870 6439 ts_tablet_manager.cc:1428] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 01:14:41.791142 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.779988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:58977 (local address 127.25.236.129:39933)
0806 01:14:41.780385 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.780392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.781845 (+ 1453us) server_negotiation.cc:407] Connection header received
0806 01:14:41.782042 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.782048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.782122 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.782538 (+ 416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:41.783491 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.784809 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.788431 (+ 3622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.789846 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.789981 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.790042 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.790698 (+ 656us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.790845 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245}
I20260806 01:14:41.791982 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.780158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:33423 (local address 127.25.236.130:35299)
0806 01:14:41.780385 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.780395 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.782923 (+ 2528us) server_negotiation.cc:407] Connection header received
0806 01:14:41.783317 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.783322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.783408 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.783477 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:14:41.784367 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.785457 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.788334 (+ 2877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.789635 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.789812 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.789890 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.791547 (+ 1657us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.791684 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:14:41.792088 6520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.778868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:39933 (local address 127.25.236.131:58977)
0806 01:14:41.781747 (+ 2879us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.781769 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.781916 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.782962 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.782968 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.783009 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:41.783297 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.783310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.785053 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.785059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.787331 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.787350 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.790289 (+ 2939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.790297 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.790483 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.790517 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.791540 (+ 1023us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.791685 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2734,"mutex_wait_us":54,"spinlock_wait_cycles":17920,"thread_start_us":3843,"threads_started":1}
I20260806 01:14:41.792654 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.779451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:35299 (local address 127.25.236.131:33423)
0806 01:14:41.782813 (+ 3362us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.782833 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.783112 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.783736 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.783742 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.783771 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:14:41.784174 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.784189 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.785723 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.785728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.788168 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.788180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.790732 (+ 2552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.791156 (+ 424us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.791363 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.791393 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.792172 (+ 779us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.792318 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3229,"mutex_wait_us":1,"spinlock_wait_cycles":5248,"thread_start_us":2002,"threads_started":1}
I20260806 01:14:41.792913 5894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e737dc1b2d64fa2a01e16de302e1fc2" candidate_uuid: "c6a00b734a74430280fe014db9169c52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e254ab43ccf04cde8655ee5c07644be6" is_pre_election: true
I20260806 01:14:41.793146 5894 raft_consensus.cc:2454] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6a00b734a74430280fe014db9169c52 in term 0.
I20260806 01:14:41.793340 6117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e737dc1b2d64fa2a01e16de302e1fc2" candidate_uuid: "c6a00b734a74430280fe014db9169c52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cab846ae46f44ec800e3312be75bca8" is_pre_election: true
I20260806 01:14:41.793596 6117 raft_consensus.cc:2454] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6a00b734a74430280fe014db9169c52 in term 0.
I20260806 01:14:41.794064 6222 leader_election.cc:302] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6a00b734a74430280fe014db9169c52, e254ab43ccf04cde8655ee5c07644be6; no voters:
I20260806 01:14:41.794849 6457 raft_consensus.cc:2781] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:14:41.795003 6457 raft_consensus.cc:482] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:14:41.795200 6457 raft_consensus.cc:3037] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:41.797467 6457 raft_consensus.cc:504] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.798533 6457 leader_election.cc:288] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [CANDIDATE]: Term 1 election: Requested vote from peers e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933), 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130:35299)
I20260806 01:14:41.799080 5894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e737dc1b2d64fa2a01e16de302e1fc2" candidate_uuid: "c6a00b734a74430280fe014db9169c52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e254ab43ccf04cde8655ee5c07644be6"
I20260806 01:14:41.799079 6117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e737dc1b2d64fa2a01e16de302e1fc2" candidate_uuid: "c6a00b734a74430280fe014db9169c52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cab846ae46f44ec800e3312be75bca8"
I20260806 01:14:41.799417 6117 raft_consensus.cc:3037] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:41.799431 5894 raft_consensus.cc:3037] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:41.802601 5894 raft_consensus.cc:2454] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6a00b734a74430280fe014db9169c52 in term 1.
I20260806 01:14:41.803076 6222 leader_election.cc:302] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6a00b734a74430280fe014db9169c52, e254ab43ccf04cde8655ee5c07644be6; no voters:
I20260806 01:14:41.803416 6457 raft_consensus.cc:2781] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:41.803439 6117 raft_consensus.cc:2454] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6a00b734a74430280fe014db9169c52 in term 1.
I20260806 01:14:41.803934 6457 raft_consensus.cc:686] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [term 1 LEADER]: Becoming Leader. State: Replica: c6a00b734a74430280fe014db9169c52, State: Running, Role: LEADER
I20260806 01:14:41.804489 6457 consensus_queue.cc:237] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:41.807228 6521 tablet_replica.cc:439] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52: TxnStatusTablet state changed. Reason: New leader c6a00b734a74430280fe014db9169c52. Latest consensus state: current_term: 1 leader_uuid: "c6a00b734a74430280fe014db9169c52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } } }
I20260806 01:14:41.807714 6521 tablet_replica.cc:442] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:14:41.808641 6524 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 01:14:41.808899 6524 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 01:14:41.809119 5643 catalog_manager.cc:5495] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 reported cstate change: term changed from 0 to 1, leader changed from <none> to c6a00b734a74430280fe014db9169c52 (127.25.236.131). New cstate: current_term: 1 leader_uuid: "c6a00b734a74430280fe014db9169c52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } health_report { overall_health: HEALTHY } } }
I20260806 01:14:41.851226 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.842345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:38439 (local address 127.0.0.1:56184)
0806 01:14:41.842607 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.842628 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.842779 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.843624 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.843630 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.843643 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:14:41.843839 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.843849 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.845348 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.845354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.846987 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.847001 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.848364 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.848370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.848588 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.848677 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:14:41.850502 (+ 1825us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:14:41.850533 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.850641 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.850843 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 01:14:41.851955 6435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.842417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56184 (local address 127.25.236.131:38439)
0806 01:14:41.842626 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.842633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.842951 (+ 318us) server_negotiation.cc:407] Connection header received
0806 01:14:41.843053 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.843058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.843283 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.843448 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:14:41.844037 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.845190 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.847179 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.847900 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.848175 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.848938 (+ 763us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:14:41.850177 (+ 1239us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:14:41.850433 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.851501 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.851604 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 01:14:41.857609 6117 raft_consensus.cc:1261] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Refusing update from remote peer c6a00b734a74430280fe014db9169c52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:41.857645 5894 raft_consensus.cc:1261] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Refusing update from remote peer c6a00b734a74430280fe014db9169c52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:41.858534 6457 consensus_queue.cc:1035] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:41.858867 6521 consensus_queue.cc:1035] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:41.872309 6456 tablet_replica.cc:439] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6: TxnStatusTablet state changed. Reason: New leader c6a00b734a74430280fe014db9169c52. Latest consensus state: current_term: 1 leader_uuid: "c6a00b734a74430280fe014db9169c52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } } }
I20260806 01:14:41.872699 6456 tablet_replica.cc:442] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:14:41.875263 6458 tablet_replica.cc:439] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8: TxnStatusTablet state changed. Reason: New leader c6a00b734a74430280fe014db9169c52. Latest consensus state: current_term: 1 leader_uuid: "c6a00b734a74430280fe014db9169c52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } } }
I20260806 01:14:41.875618 6458 tablet_replica.cc:442] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:14:41.879890 6521 tablet_replica.cc:439] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6a00b734a74430280fe014db9169c52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } } }
I20260806 01:14:41.880200 6521 tablet_replica.cc:442] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:14:41.880406 6466 mvcc.cc:205] Tried to move back new op lower bound from 7315369500079558656 to 7315369499876052992. Current Snapshot: MvccSnapshot[applied={T|T < 7315369500079558656 or (T in {7315369500079558656})}]
I20260806 01:14:41.889423 6456 tablet_replica.cc:439] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6a00b734a74430280fe014db9169c52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } } }
I20260806 01:14:41.889744 6456 tablet_replica.cc:442] T 2e737dc1b2d64fa2a01e16de302e1fc2 P e254ab43ccf04cde8655ee5c07644be6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:14:41.896034 6457 tablet_replica.cc:439] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6a00b734a74430280fe014db9169c52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } } }
I20260806 01:14:41.896705 6457 tablet_replica.cc:442] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:14:41.897307 6465 mvcc.cc:205] Tried to move back new op lower bound from 7315369500079558656 to 7315369499876052992. Current Snapshot: MvccSnapshot[applied={T|T < 7315369500079558656}]
I20260806 01:14:41.902824 6458 tablet_replica.cc:439] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6a00b734a74430280fe014db9169c52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } } }
I20260806 01:14:41.903208 6458 tablet_replica.cc:442] T 2e737dc1b2d64fa2a01e16de302e1fc2 P 1cab846ae46f44ec800e3312be75bca8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:14:41.913087 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.895630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.25.236.130:35299)
0806 01:14:41.895835 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.895841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.895864 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:14:41.905312 (+ 9448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.905319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.905467 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.905628 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:14:41.906608 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.907801 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.910112 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.910800 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.910942 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.911569 (+ 627us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:14:41.912353 (+ 784us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:14:41.912503 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.912670 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.912765 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:14:41.936122 6540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.927691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:38439 (local address 127.25.236.130:39599)
0806 01:14:41.928324 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:41.928349 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:41.928540 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:41.928949 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:41.928954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:41.928969 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:14:41.929352 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.929364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.931039 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.931045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.932897 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:41.932914 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.934213 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.934220 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:41.934449 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.934527 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:14:41.935451 (+ 924us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:14:41.935467 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 01:14:41.935536 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 01:14:41.935710 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":289,"threads_started":1}
I20260806 01:14:41.936049 6435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:41.927847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:39599 (local address 127.25.236.131:38439)
0806 01:14:41.928073 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:41.928079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:41.928455 (+ 376us) server_negotiation.cc:407] Connection header received
0806 01:14:41.928714 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:41.928720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:41.928808 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:41.928907 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:14:41.929528 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.930862 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.933178 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:41.934014 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:41.934158 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:41.935037 (+ 879us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:14:41.935268 (+ 231us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:14:41.935429 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:41.935616 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 01:14:41.935709 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 01:14:41.949095 6329 raft_consensus.cc:1261] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Refusing update from remote peer 1cab846ae46f44ec800e3312be75bca8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:41.949198 5894 raft_consensus.cc:3037] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:41.949972 6494 consensus_queue.cc:1035] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:41.952723 5894 raft_consensus.cc:1261] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Refusing update from remote peer 1cab846ae46f44ec800e3312be75bca8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:41.953382 6494 consensus_queue.cc:1035] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:42.028620 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:42.022582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:38903 (local address 127.25.236.130:35299)
0806 01:14:42.022789 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:42.022797 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:42.023282 (+ 485us) server_negotiation.cc:407] Connection header received
0806 01:14:42.023451 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:42.023456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:42.023546 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:42.023678 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:14:42.024310 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.025015 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.026813 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.027277 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.027423 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:42.027873 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:14:42.027973 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:14:42.028072 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:42.028217 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 01:14:42.028298 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:14:42.028837 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:42.022429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:35299 (local address 127.25.236.131:38903)
0806 01:14:42.023170 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:42.023187 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:42.023315 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:42.023730 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:42.023734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:42.023745 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:14:42.024149 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:42.024161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.025151 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.025158 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:42.026668 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:42.026681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.027471 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.027480 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:42.027628 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:42.027685 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:14:42.028076 (+ 391us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:14:42.028090 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 01:14:42.028194 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 01:14:42.028336 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":392,"threads_started":1}
I20260806 01:14:42.075855 5643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:42.076109 5643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:42.079870 5643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615: Sending DeleteTablet for 3 replicas of tablet 265514c867c5477ba1d17a60e29e55c4
I20260806 01:14:42.081189 5864 tablet_service.cc:1514] Processing DeleteTablet for tablet 265514c867c5477ba1d17a60e29e55c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:42 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:54890
I20260806 01:14:42.081620 6294 tablet_service.cc:1514] Processing DeleteTablet for tablet 265514c867c5477ba1d17a60e29e55c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:42 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:56178
I20260806 01:14:42.082809 6088 tablet_service.cc:1514] Processing DeleteTablet for tablet 265514c867c5477ba1d17a60e29e55c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:42 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:43124
I20260806 01:14:42.098711 6561 tablet_replica.cc:330] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6: stopping tablet replica
I20260806 01:14:42.098999 6561 raft_consensus.cc:2229] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:42.099300 6561 raft_consensus.cc:2258] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:42.100024 6563 tablet_replica.cc:330] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52: stopping tablet replica
I20260806 01:14:42.100284 6563 raft_consensus.cc:2229] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:42.100633 6563 raft_consensus.cc:2258] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:42.101329 6564 tablet_replica.cc:330] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8: stopping tablet replica
I20260806 01:14:42.101559 6564 raft_consensus.cc:2229] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:14:42.102095 6564 raft_consensus.cc:2258] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:42.103272 6561 ts_tablet_manager.cc:1905] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:42.105914 6564 ts_tablet_manager.cc:1905] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:42.107126 6563 ts_tablet_manager.cc:1905] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:42.118737 6564 ts_tablet_manager.cc:1918] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:14:42.118746 6561 ts_tablet_manager.cc:1918] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:14:42.118896 6561 log.cc:1196] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/265514c867c5477ba1d17a60e29e55c4
I20260806 01:14:42.119113 6564 log.cc:1196] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/265514c867c5477ba1d17a60e29e55c4
I20260806 01:14:42.119705 6564 ts_tablet_manager.cc:1939] T 265514c867c5477ba1d17a60e29e55c4 P 1cab846ae46f44ec800e3312be75bca8: Deleting consensus metadata
I20260806 01:14:42.121021 5619 catalog_manager.cc:4841] TS 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130:35299): tablet 265514c867c5477ba1d17a60e29e55c4 (table test-table [id=88550a24f17048f6bfdb29513f4d00f8]) successfully deleted
I20260806 01:14:42.123414 6561 ts_tablet_manager.cc:1939] T 265514c867c5477ba1d17a60e29e55c4 P e254ab43ccf04cde8655ee5c07644be6: Deleting consensus metadata
I20260806 01:14:42.124965 5618 catalog_manager.cc:4841] TS e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933): tablet 265514c867c5477ba1d17a60e29e55c4 (table test-table [id=88550a24f17048f6bfdb29513f4d00f8]) successfully deleted
I20260806 01:14:42.128752 6563 ts_tablet_manager.cc:1918] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:14:42.129091 6563 log.cc:1196] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/265514c867c5477ba1d17a60e29e55c4
I20260806 01:14:42.129693 6563 ts_tablet_manager.cc:1939] T 265514c867c5477ba1d17a60e29e55c4 P c6a00b734a74430280fe014db9169c52: Deleting consensus metadata
I20260806 01:14:42.131438 5621 catalog_manager.cc:4841] TS c6a00b734a74430280fe014db9169c52 (127.25.236.131:38439): tablet 265514c867c5477ba1d17a60e29e55c4 (table test-table [id=88550a24f17048f6bfdb29513f4d00f8]) successfully deleted
Aug 06 01:14:42 dist-test-slave-7nm7 krb5kdc[5578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:14:42 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:42 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:14:42.207436 6512 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:42.191488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.25.236.190:42737)
0806 01:14:42.191735 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:42.191744 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:42.192143 (+ 399us) server_negotiation.cc:407] Connection header received
0806 01:14:42.192388 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:42.192397 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:42.192491 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:42.192598 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:42.194721 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.195917 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.198198 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.198983 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.199101 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:42.207071 (+ 7970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49196: BlockingRecv error: recv got EOF from 127.0.0.1:49196 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 01:14:42 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM
I20260806 01:14:42.250563 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:42.227775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49204 (local address 127.25.236.190:42737)
0806 01:14:42.227978 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:42.227987 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:42.228752 (+ 765us) server_negotiation.cc:407] Connection header received
0806 01:14:42.229008 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:42.229017 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:42.229117 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:42.229216 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:42.231342 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.232708 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.235250 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.236028 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.236152 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:42.242437 (+ 6285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:42.242446 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:42.242454 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:42.242505 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:42.246696 (+ 4191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:42.247659 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:42.247668 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:42.247671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:42.247836 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:42.248384 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:42.248391 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:42.248397 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:42.248826 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:42.249133 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:42.249746 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 01:14:42.250059 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 01:14:42.254808 5643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:14:42.255467 5643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:14:42.263015 6088 tablet_service.cc:1467] Processing CreateTablet for tablet 9d22304787c54cda8c7e3e8614126592 (DEFAULT_TABLE table=test-table [id=ef12543149094408a477d3d6ebc7f259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:42.263015 5864 tablet_service.cc:1467] Processing CreateTablet for tablet 9d22304787c54cda8c7e3e8614126592 (DEFAULT_TABLE table=test-table [id=ef12543149094408a477d3d6ebc7f259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:42.263473 5864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d22304787c54cda8c7e3e8614126592. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:42.263485 6088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d22304787c54cda8c7e3e8614126592. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:42.270583 6294 tablet_service.cc:1467] Processing CreateTablet for tablet 9d22304787c54cda8c7e3e8614126592 (DEFAULT_TABLE table=test-table [id=ef12543149094408a477d3d6ebc7f259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:42.271095 6294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d22304787c54cda8c7e3e8614126592. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:42.271205 6440 tablet_bootstrap.cc:492] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8: Bootstrap starting.
I20260806 01:14:42.272948 6440 tablet_bootstrap.cc:654] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:42.278605 6439 tablet_bootstrap.cc:492] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6: Bootstrap starting.
I20260806 01:14:42.280910 6439 tablet_bootstrap.cc:654] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:42.285395 6439 tablet_bootstrap.cc:492] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6: No bootstrap required, opened a new log
I20260806 01:14:42.285655 6439 ts_tablet_manager.cc:1397] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260806 01:14:42.286829 6439 raft_consensus.cc:348] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.287266 6439 raft_consensus.cc:374] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:42.287426 6439 raft_consensus.cc:729] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e254ab43ccf04cde8655ee5c07644be6, State: Initialized, Role: FOLLOWER
I20260806 01:14:42.287796 6439 consensus_queue.cc:260] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.291476 6439 ts_tablet_manager.cc:1428] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 01:14:42.295032 6593 tablet_bootstrap.cc:492] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52: Bootstrap starting.
I20260806 01:14:42.296634 6440 tablet_bootstrap.cc:492] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8: No bootstrap required, opened a new log
I20260806 01:14:42.296830 6440 ts_tablet_manager.cc:1397] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.009s
I20260806 01:14:42.297596 6440 raft_consensus.cc:348] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.297859 6440 raft_consensus.cc:374] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:42.297968 6440 raft_consensus.cc:729] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cab846ae46f44ec800e3312be75bca8, State: Initialized, Role: FOLLOWER
I20260806 01:14:42.298282 6440 consensus_queue.cc:260] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.298887 6440 ts_tablet_manager.cc:1428] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 01:14:42.302534 6593 tablet_bootstrap.cc:654] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:42.304912 6593 tablet_bootstrap.cc:492] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52: No bootstrap required, opened a new log
I20260806 01:14:42.305343 6593 ts_tablet_manager.cc:1397] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.004s
I20260806 01:14:42.306376 6593 raft_consensus.cc:348] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.306731 6593 raft_consensus.cc:374] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:42.306872 6593 raft_consensus.cc:729] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6a00b734a74430280fe014db9169c52, State: Initialized, Role: FOLLOWER
I20260806 01:14:42.307140 6593 consensus_queue.cc:260] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.307683 6593 ts_tablet_manager.cc:1428] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 01:14:42.525413 6458 raft_consensus.cc:482] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:14:42.525681 6458 raft_consensus.cc:504] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.526629 6458 leader_election.cc:288] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933), c6a00b734a74430280fe014db9169c52 (127.25.236.131:38439)
I20260806 01:14:42.527525 6325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d22304787c54cda8c7e3e8614126592" candidate_uuid: "1cab846ae46f44ec800e3312be75bca8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6a00b734a74430280fe014db9169c52" is_pre_election: true
I20260806 01:14:42.527546 5893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d22304787c54cda8c7e3e8614126592" candidate_uuid: "1cab846ae46f44ec800e3312be75bca8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e254ab43ccf04cde8655ee5c07644be6" is_pre_election: true
I20260806 01:14:42.527805 6325 raft_consensus.cc:2454] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cab846ae46f44ec800e3312be75bca8 in term 0.
I20260806 01:14:42.527865 5893 raft_consensus.cc:2454] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cab846ae46f44ec800e3312be75bca8 in term 0.
I20260806 01:14:42.528342 6007 leader_election.cc:302] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cab846ae46f44ec800e3312be75bca8, c6a00b734a74430280fe014db9169c52; no voters:
I20260806 01:14:42.528697 6458 raft_consensus.cc:2781] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:14:42.528905 6458 raft_consensus.cc:482] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:14:42.529115 6458 raft_consensus.cc:3037] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:42.531646 6458 raft_consensus.cc:504] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.532462 6458 leader_election.cc:288] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [CANDIDATE]: Term 1 election: Requested vote from peers e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933), c6a00b734a74430280fe014db9169c52 (127.25.236.131:38439)
I20260806 01:14:42.532985 5893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d22304787c54cda8c7e3e8614126592" candidate_uuid: "1cab846ae46f44ec800e3312be75bca8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e254ab43ccf04cde8655ee5c07644be6"
I20260806 01:14:42.533275 5893 raft_consensus.cc:3037] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:42.533314 6325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d22304787c54cda8c7e3e8614126592" candidate_uuid: "1cab846ae46f44ec800e3312be75bca8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6a00b734a74430280fe014db9169c52"
I20260806 01:14:42.533550 6325 raft_consensus.cc:3037] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:42.536540 5893 raft_consensus.cc:2454] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cab846ae46f44ec800e3312be75bca8 in term 1.
I20260806 01:14:42.536692 6325 raft_consensus.cc:2454] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cab846ae46f44ec800e3312be75bca8 in term 1.
I20260806 01:14:42.537070 6005 leader_election.cc:302] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cab846ae46f44ec800e3312be75bca8, e254ab43ccf04cde8655ee5c07644be6; no voters:
I20260806 01:14:42.537599 6458 raft_consensus.cc:2781] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:42.537796 6458 raft_consensus.cc:686] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [term 1 LEADER]: Becoming Leader. State: Replica: 1cab846ae46f44ec800e3312be75bca8, State: Running, Role: LEADER
I20260806 01:14:42.538139 6458 consensus_queue.cc:237] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.541652 5642 catalog_manager.cc:5495] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130). New cstate: current_term: 1 leader_uuid: "1cab846ae46f44ec800e3312be75bca8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } health_report { overall_health: UNKNOWN } } }
I20260806 01:14:42.598445 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:42.590860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43146 (local address 127.25.236.130:35299)
0806 01:14:42.591683 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:42.591688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:42.591709 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:14:42.591786 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:42.591790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:42.591860 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:42.591965 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:14:42.592478 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.593643 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.595654 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.596159 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.596440 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:42.597161 (+ 721us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:14:42.597387 (+ 226us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:14:42.597544 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:42.597786 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 01:14:42.597861 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":521,"threads_started":1}
I20260806 01:14:42.601948 6325 raft_consensus.cc:1261] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Refusing update from remote peer 1cab846ae46f44ec800e3312be75bca8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:42.602034 5893 raft_consensus.cc:1261] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Refusing update from remote peer 1cab846ae46f44ec800e3312be75bca8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:42.602907 6494 consensus_queue.cc:1035] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:42.603381 6458 consensus_queue.cc:1035] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:42.614717 6602 mvcc.cc:205] Tried to move back new op lower bound from 7315369503130779648 to 7315369502879391744. Current Snapshot: MvccSnapshot[applied={T|T < 7315369503130779648}]
I20260806 01:14:42.630565 5642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:42.630908 5642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:42.636996 5642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615: Sending DeleteTablet for 3 replicas of tablet 9d22304787c54cda8c7e3e8614126592
I20260806 01:14:42.638091 6088 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d22304787c54cda8c7e3e8614126592 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:42 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:43124
I20260806 01:14:42.638283 6086 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d22304787c54cda8c7e3e8614126592 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:42 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:43124
I20260806 01:14:42.638305 6294 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d22304787c54cda8c7e3e8614126592 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:42 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:56178
I20260806 01:14:42.638963 5864 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d22304787c54cda8c7e3e8614126592 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:42 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:54890
I20260806 01:14:42.639683 6612 tablet_replica.cc:330] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8: stopping tablet replica
I20260806 01:14:42.639729 5642 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 01:14:42.640141 6614 tablet_replica.cc:330] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52: stopping tablet replica
I20260806 01:14:42.640336 6614 raft_consensus.cc:2229] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 01:14:42.640435 5642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:14:42.640602 6614 raft_consensus.cc:2258] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:42.640893 6612 raft_consensus.cc:2229] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:14:42.641358 6612 raft_consensus.cc:2258] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:42.642655 6612 ts_tablet_manager.cc:1905] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:42.650430 5862 tablet_service.cc:1467] Processing CreateTablet for tablet 6f367e150a87472195087403c31eb829 (DEFAULT_TABLE table=test-table [id=f4277a19e55148f385088679d68bba2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:42.650584 6295 tablet_service.cc:1467] Processing CreateTablet for tablet 6f367e150a87472195087403c31eb829 (DEFAULT_TABLE table=test-table [id=f4277a19e55148f385088679d68bba2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:42.650949 5862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f367e150a87472195087403c31eb829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:42.651038 6295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f367e150a87472195087403c31eb829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:42.655480 6088 tablet_service.cc:1467] Processing CreateTablet for tablet 6f367e150a87472195087403c31eb829 (DEFAULT_TABLE table=test-table [id=f4277a19e55148f385088679d68bba2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:14:42.656535 6088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f367e150a87472195087403c31eb829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:42.659834 6614 ts_tablet_manager.cc:1905] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:42.663060 6593 tablet_bootstrap.cc:492] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52: Bootstrap starting.
I20260806 01:14:42.665160 6593 tablet_bootstrap.cc:654] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:42.673009 6612 ts_tablet_manager.cc:1918] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:42.673527 6439 tablet_bootstrap.cc:492] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6: Bootstrap starting.
I20260806 01:14:42.676533 6439 tablet_bootstrap.cc:654] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:42.680146 6614 ts_tablet_manager.cc:1918] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:42.680346 6614 log.cc:1196] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/9d22304787c54cda8c7e3e8614126592
I20260806 01:14:42.680941 6614 ts_tablet_manager.cc:1939] T 9d22304787c54cda8c7e3e8614126592 P c6a00b734a74430280fe014db9169c52: Deleting consensus metadata
I20260806 01:14:42.681360 6593 tablet_bootstrap.cc:492] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52: No bootstrap required, opened a new log
I20260806 01:14:42.681541 6593 ts_tablet_manager.cc:1397] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.008s
I20260806 01:14:42.682467 5621 catalog_manager.cc:4841] TS c6a00b734a74430280fe014db9169c52 (127.25.236.131:38439): tablet 9d22304787c54cda8c7e3e8614126592 (table test-table [id=ef12543149094408a477d3d6ebc7f259]) successfully deleted
I20260806 01:14:42.682392 6593 raft_consensus.cc:348] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.682698 6593 raft_consensus.cc:374] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:42.682793 6593 raft_consensus.cc:729] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6a00b734a74430280fe014db9169c52, State: Initialized, Role: FOLLOWER
I20260806 01:14:42.683089 6616 tablet_replica.cc:330] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6: stopping tablet replica
I20260806 01:14:42.683105 6612 log.cc:1196] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/9d22304787c54cda8c7e3e8614126592
I20260806 01:14:42.683084 6593 consensus_queue.cc:260] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.683331 6616 raft_consensus.cc:2229] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:42.683620 6616 raft_consensus.cc:2258] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:42.683692 6593 ts_tablet_manager.cc:1428] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 01:14:42.684803 6616 ts_tablet_manager.cc:1905] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:42.684839 6612 ts_tablet_manager.cc:1939] T 9d22304787c54cda8c7e3e8614126592 P 1cab846ae46f44ec800e3312be75bca8: Deleting consensus metadata
I20260806 01:14:42.686066 5619 catalog_manager.cc:4841] TS 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130:35299): tablet 9d22304787c54cda8c7e3e8614126592 (table test-table [id=ef12543149094408a477d3d6ebc7f259]) successfully deleted
I20260806 01:14:42.686584 6440 tablet_bootstrap.cc:492] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8: Bootstrap starting.
I20260806 01:14:42.688459 6440 tablet_bootstrap.cc:654] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:42.691959 6616 ts_tablet_manager.cc:1918] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:14:42.692081 6440 tablet_bootstrap.cc:492] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8: No bootstrap required, opened a new log
I20260806 01:14:42.692231 6616 log.cc:1196] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/9d22304787c54cda8c7e3e8614126592
I20260806 01:14:42.692341 6440 ts_tablet_manager.cc:1397] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 01:14:42.692791 6616 ts_tablet_manager.cc:1939] T 9d22304787c54cda8c7e3e8614126592 P e254ab43ccf04cde8655ee5c07644be6: Deleting consensus metadata
W20260806 01:14:42.693187 5619 catalog_manager.cc:4808] TS 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130:35299): delete failed for tablet 9d22304787c54cda8c7e3e8614126592 because the tablet was not found. No further retry: Not found: Tablet not found: 9d22304787c54cda8c7e3e8614126592
I20260806 01:14:42.693363 6440 raft_consensus.cc:348] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.693725 6440 raft_consensus.cc:374] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:42.693853 6440 raft_consensus.cc:729] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cab846ae46f44ec800e3312be75bca8, State: Initialized, Role: FOLLOWER
I20260806 01:14:42.694221 5618 catalog_manager.cc:4841] TS e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933): tablet 9d22304787c54cda8c7e3e8614126592 (table test-table [id=ef12543149094408a477d3d6ebc7f259]) successfully deleted
I20260806 01:14:42.694260 6440 consensus_queue.cc:260] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.696568 6439 tablet_bootstrap.cc:492] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6: No bootstrap required, opened a new log
I20260806 01:14:42.696805 6439 ts_tablet_manager.cc:1397] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.005s
I20260806 01:14:42.697347 6440 ts_tablet_manager.cc:1428] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260806 01:14:42.697860 6439 raft_consensus.cc:348] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.698158 6439 raft_consensus.cc:374] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:42.698290 6439 raft_consensus.cc:729] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e254ab43ccf04cde8655ee5c07644be6, State: Initialized, Role: FOLLOWER
I20260806 01:14:42.698921 6439 consensus_queue.cc:260] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.699570 6439 ts_tablet_manager.cc:1428] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 01:14:42.776732 6544 raft_consensus.cc:482] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:14:42.776922 6544 raft_consensus.cc:504] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.777697 6544 leader_election.cc:288] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933), 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130:35299)
I20260806 01:14:42.778378 5893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f367e150a87472195087403c31eb829" candidate_uuid: "c6a00b734a74430280fe014db9169c52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e254ab43ccf04cde8655ee5c07644be6" is_pre_election: true
I20260806 01:14:42.778524 6117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f367e150a87472195087403c31eb829" candidate_uuid: "c6a00b734a74430280fe014db9169c52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cab846ae46f44ec800e3312be75bca8" is_pre_election: true
I20260806 01:14:42.778745 5893 raft_consensus.cc:2454] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6a00b734a74430280fe014db9169c52 in term 0.
I20260806 01:14:42.778805 6117 raft_consensus.cc:2454] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6a00b734a74430280fe014db9169c52 in term 0.
I20260806 01:14:42.779284 6222 leader_election.cc:302] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6a00b734a74430280fe014db9169c52, e254ab43ccf04cde8655ee5c07644be6; no voters:
I20260806 01:14:42.779582 6544 raft_consensus.cc:2781] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:14:42.779680 6544 raft_consensus.cc:482] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:14:42.779795 6544 raft_consensus.cc:3037] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:42.781991 6544 raft_consensus.cc:504] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.782634 6544 leader_election.cc:288] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [CANDIDATE]: Term 1 election: Requested vote from peers e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933), 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130:35299)
I20260806 01:14:42.783093 5893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f367e150a87472195087403c31eb829" candidate_uuid: "c6a00b734a74430280fe014db9169c52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e254ab43ccf04cde8655ee5c07644be6"
I20260806 01:14:42.783341 5893 raft_consensus.cc:3037] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:42.783342 6117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f367e150a87472195087403c31eb829" candidate_uuid: "c6a00b734a74430280fe014db9169c52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cab846ae46f44ec800e3312be75bca8"
I20260806 01:14:42.783617 6117 raft_consensus.cc:3037] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:42.785885 6117 raft_consensus.cc:2454] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6a00b734a74430280fe014db9169c52 in term 1.
I20260806 01:14:42.786059 5893 raft_consensus.cc:2454] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6a00b734a74430280fe014db9169c52 in term 1.
I20260806 01:14:42.786422 6223 leader_election.cc:302] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cab846ae46f44ec800e3312be75bca8, c6a00b734a74430280fe014db9169c52; no voters:
I20260806 01:14:42.786835 6544 raft_consensus.cc:2781] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:42.787353 6544 raft_consensus.cc:686] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [term 1 LEADER]: Becoming Leader. State: Replica: c6a00b734a74430280fe014db9169c52, State: Running, Role: LEADER
I20260806 01:14:42.787689 6544 consensus_queue.cc:237] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } }
I20260806 01:14:42.790642 5642 catalog_manager.cc:5495] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 reported cstate change: term changed from 0 to 1, leader changed from <none> to c6a00b734a74430280fe014db9169c52 (127.25.236.131). New cstate: current_term: 1 leader_uuid: "c6a00b734a74430280fe014db9169c52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6a00b734a74430280fe014db9169c52" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 38439 } health_report { overall_health: HEALTHY } } }
I20260806 01:14:42.819010 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:42.812974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56200 (local address 127.25.236.131:38439)
0806 01:14:42.813710 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:42.813715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:42.813737 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:14:42.813793 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:42.813797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:42.813861 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:42.813957 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:14:42.814576 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.815594 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.816995 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.817499 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.817609 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:42.818091 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:14:42.818359 (+ 268us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:14:42.818462 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:42.818635 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 01:14:42.818707 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":464,"threads_started":1}
I20260806 01:14:42.835963 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:42.829266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:34891 (local address 127.25.236.131:38439)
0806 01:14:42.829519 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:42.829526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:42.829907 (+ 381us) server_negotiation.cc:407] Connection header received
0806 01:14:42.830134 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:42.830138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:42.830210 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:42.830355 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:14:42.830918 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.831650 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.833702 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.834296 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.834407 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:42.835254 (+ 847us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:14:42.835372 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:14:42.835452 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:42.835609 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 01:14:42.835716 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":105}
I20260806 01:14:42.836256 6636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:42.829104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:38439 (local address 127.25.236.131:34891)
0806 01:14:42.829771 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:42.829797 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:42.829959 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:42.830389 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:42.830394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:42.830408 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:14:42.830736 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:42.830748 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.831799 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.831804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:42.833534 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:42.833551 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.834622 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.834633 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:42.834989 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:42.835044 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:14:42.835481 (+ 437us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:14:42.835493 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 01:14:42.835582 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 01:14:42.835770 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":4,"thread_start_us":315,"threads_started":1}
I20260806 01:14:42.845772 5893 raft_consensus.cc:1261] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Refusing update from remote peer c6a00b734a74430280fe014db9169c52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:42.845748 6117 raft_consensus.cc:1261] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Refusing update from remote peer c6a00b734a74430280fe014db9169c52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:14:42.846503 6637 consensus_queue.cc:1035] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cab846ae46f44ec800e3312be75bca8" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:42.846855 6629 consensus_queue.cc:1035] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e254ab43ccf04cde8655ee5c07644be6" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:14:42.909617 5642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:42.909905 5642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:14:42.913398 5642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 175df273e91c468eae0d35a24f131615: Sending DeleteTablet for 3 replicas of tablet 6f367e150a87472195087403c31eb829
I20260806 01:14:42.914345 5864 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f367e150a87472195087403c31eb829 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:42 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:54890
I20260806 01:14:42.914564 6088 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f367e150a87472195087403c31eb829 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:42 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:43124
I20260806 01:14:42.914664 6616 tablet_replica.cc:330] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6: stopping tablet replica
I20260806 01:14:42.914603 6295 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f367e150a87472195087403c31eb829 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:14:42 UTC) from {username='oryx', principal='oryx/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:56178
I20260806 01:14:42.914834 6612 tablet_replica.cc:330] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8: stopping tablet replica
I20260806 01:14:42.914881 6614 tablet_replica.cc:330] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52: stopping tablet replica
I20260806 01:14:42.914875 6616 raft_consensus.cc:2229] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:42.915000 6612 raft_consensus.cc:2229] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:14:42.915040 6614 raft_consensus.cc:2229] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:14:42.915072 6616 raft_consensus.cc:2258] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:42.915195 6612 raft_consensus.cc:2258] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:42.915566 6614 raft_consensus.cc:2258] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:14:42.916842 6616 ts_tablet_manager.cc:1905] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:42.916841 6612 ts_tablet_manager.cc:1905] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:42.919744 6614 ts_tablet_manager.cc:1905] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:14:42.924613 6612 ts_tablet_manager.cc:1918] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:14:42.924865 6612 log.cc:1196] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/6f367e150a87472195087403c31eb829
I20260806 01:14:42.925307 6612 ts_tablet_manager.cc:1939] T 6f367e150a87472195087403c31eb829 P 1cab846ae46f44ec800e3312be75bca8: Deleting consensus metadata
I20260806 01:14:42.925668 6616 ts_tablet_manager.cc:1918] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:14:42.925896 6616 log.cc:1196] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/6f367e150a87472195087403c31eb829
I20260806 01:14:42.926465 6616 ts_tablet_manager.cc:1939] T 6f367e150a87472195087403c31eb829 P e254ab43ccf04cde8655ee5c07644be6: Deleting consensus metadata
I20260806 01:14:42.926538 5619 catalog_manager.cc:4841] TS 1cab846ae46f44ec800e3312be75bca8 (127.25.236.130:35299): tablet 6f367e150a87472195087403c31eb829 (table test-table [id=f4277a19e55148f385088679d68bba2d]) successfully deleted
I20260806 01:14:42.927007 6614 ts_tablet_manager.cc:1918] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:14:42.927194 6614 log.cc:1196] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/6f367e150a87472195087403c31eb829
I20260806 01:14:42.927882 6614 ts_tablet_manager.cc:1939] T 6f367e150a87472195087403c31eb829 P c6a00b734a74430280fe014db9169c52: Deleting consensus metadata
I20260806 01:14:42.927892 5618 catalog_manager.cc:4841] TS e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933): tablet 6f367e150a87472195087403c31eb829 (table test-table [id=f4277a19e55148f385088679d68bba2d]) successfully deleted
I20260806 01:14:42.929100 5621 catalog_manager.cc:4841] TS c6a00b734a74430280fe014db9169c52 (127.25.236.131:38439): tablet 6f367e150a87472195087403c31eb829 (table test-table [id=f4277a19e55148f385088679d68bba2d]) successfully deleted
I20260806 01:14:42.929288 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 5758
Aug 06 01:14:42 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.236.129@KRBTEST.COM
I20260806 01:14:42.950541 6644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:42.930188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54918 (local address 127.25.236.129:39933)
0806 01:14:42.931581 (+ 1393us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:42.931586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:42.931604 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:14:42.931656 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:42.931660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:42.931720 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:42.931822 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:42.933648 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.934652 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.936543 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:42.936952 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:42.937056 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:42.942426 (+ 5370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:42.942445 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:42.942470 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:42.942530 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:42.947069 (+ 4539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:42.948005 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:42.948016 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:42.948020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:42.948219 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:42.948639 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:42.948645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:42.948649 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:42.949137 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:42.949475 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:42.949784 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 01:14:42.950039 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1278,"thread_start_us":1173,"threads_started":1}
I20260806 01:14:42.951416 5967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:43.089082 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 5758
I20260806 01:14:43.129117 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 5981
Aug 06 01:14:43 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.236.130@KRBTEST.COM
I20260806 01:14:43.151628 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:43.129998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43160 (local address 127.25.236.130:35299)
0806 01:14:43.130634 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:43.130641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:43.130662 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:14:43.130764 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:43.130770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:43.130852 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:43.130972 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:43.132899 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:43.134116 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:43.136022 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:43.136623 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:43.136760 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:43.143513 (+ 6753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:43.143535 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:43.143556 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:43.143611 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:43.147955 (+ 4344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:43.148971 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:43.148989 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:43.148994 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:43.149194 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:43.149671 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:43.149677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:43.149681 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:43.150202 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:43.150584 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:43.150918 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 01:14:43.151204 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":406,"threads_started":1}
I20260806 01:14:43.152423 6187 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:43.291242 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 5981
I20260806 01:14:43.327157 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 6198
I20260806 01:14:43.334851 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:43.333612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:39933 (local address 127.25.236.131:51129)
0806 01:14:43.334198 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:43.334437 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:39933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":947,"threads_started":1}
W20260806 01:14:43.335168 6222 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.236.129:39933: connect: Connection refused (error 111)
I20260806 01:14:43.337145 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:43.336699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:39933 (local address 127.25.236.131:38173)
0806 01:14:43.336928 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:43.337005 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:39933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260806 01:14:43.337368 6222 consensus_peers.cc:487] T 2e737dc1b2d64fa2a01e16de302e1fc2 P c6a00b734a74430280fe014db9169c52 -> Peer e254ab43ccf04cde8655ee5c07644be6 (127.25.236.129:39933): Couldn't send request to peer e254ab43ccf04cde8655ee5c07644be6. Status: Network error: Client connection negotiation failed: client connection to 127.25.236.129:39933: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 06 01:14:43 dist-test-slave-7nm7 krb5kdc[5578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.236.131@KRBTEST.COM
I20260806 01:14:43.348501 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:43.328314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56202 (local address 127.25.236.131:38439)
0806 01:14:43.328548 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:43.328554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:43.328602 (+ 48us) server_negotiation.cc:407] Connection header received
0806 01:14:43.328790 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:43.328795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:43.328904 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:43.329015 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:43.330507 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:43.331589 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:43.333053 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:43.333667 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:43.333818 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:43.340761 (+ 6943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:43.340787 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:43.340809 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:43.340872 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:43.345117 (+ 4245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:43.345952 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:43.345965 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:43.345970 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:43.346144 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:43.346555 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:43.346559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:43.346562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:43.347050 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:43.347502 (+ 452us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:43.347737 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 01:14:43.348001 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:14:43.349278 6396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:43.511317 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 6198
I20260806 01:14:43.547493 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 5594
I20260806 01:14:43.548810 5708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:43.665485 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 5594
2026-08-06T01:14:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6645 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:14:43 dist-test-slave-7nm7 krb5kdc[6662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:14:43 dist-test-slave-7nm7 krb5kdc[6662](info): set up 2 sockets
Aug 06 01:14:43 dist-test-slave-7nm7 krb5kdc[6662](info): commencing operation
krb5kdc: starting...
W20260806 01:14:45.027490 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:14:45 dist-test-slave-7nm7 krb5kdc[6662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:14:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:14:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:14:45.334820 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:44291
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:44291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:45.704389 6685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:45.704769 6685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:45.704932 6685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:45.712682 6685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:45.712853 6685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:45.712987 6685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:45.713104 6685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:45.713198 6685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:14:45.723798 6685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:44291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:44291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:45.725279 6685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:45.726831 6685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:45.739373 6701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:45.739449 6695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:45.739471 6698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:45.740736 6685 server_base.cc:1034] running on GCE node
I20260806 01:14:45.742069 6685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:45.744215 6685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:45.745489 6685 hybrid_clock.cc:648] HybridClock initialized: now 1785978885745423 us; error 51 us; skew 500 ppm
I20260806 01:14:45.745862 6685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T01:14:45Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2149 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:14:45 dist-test-slave-7nm7 krb5kdc[6711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:14:45 dist-test-slave-7nm7 krb5kdc[6711](info): set up 2 sockets
Aug 06 01:14:45 dist-test-slave-7nm7 krb5kdc[6711](info): commencing operation
krb5kdc: starting...
W20260806 01:14:47.208711 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:14:47 dist-test-slave-7nm7 krb5kdc[6711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:14:47.514091 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:39649
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:32937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:39649
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:47.862917 6734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:47.863216 6734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:47.863395 6734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:47.871341 6734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:47.871456 6734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:47.871549 6734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:47.871656 6734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:47.871742 6734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:14:47.881903 6734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:32937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:39649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:39649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:47.883322 6734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:47.884626 6734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:47.896840 6746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:47.896953 6743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:47.896978 6748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:47.898322 6734 server_base.cc:1034] running on GCE node
I20260806 01:14:47.899626 6734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:47.901850 6734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:47.903115 6734 hybrid_clock.cc:648] HybridClock initialized: now 1785978887903031 us; error 56 us; skew 500 ppm
I20260806 01:14:47.903514 6734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:47 dist-test-slave-7nm7 krb5kdc[6711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978887, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:47.907521 6734 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:14:47.909126 6734 webserver.cc:460] Webserver started at http://127.25.236.190:46855/ using document root <none> and password file <none>
I20260806 01:14:47.909968 6734 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:47.910123 6734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:47.910504 6734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:47.913796 6734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "aabd3ef1411f472c9ecac78f1dcbdf99"
format_stamp: "Formatted at 2026-08-06 01:14:47 on dist-test-slave-7nm7"
server_key: "b430b24b4926f0f3925cd595cdcb8c30"
server_key_iv: "94074bec3e8c4df3917d7772e7ef8b1e"
server_key_version: "encryptionkey@0"
I20260806 01:14:47.914633 6734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "aabd3ef1411f472c9ecac78f1dcbdf99"
format_stamp: "Formatted at 2026-08-06 01:14:47 on dist-test-slave-7nm7"
server_key: "b430b24b4926f0f3925cd595cdcb8c30"
server_key_iv: "94074bec3e8c4df3917d7772e7ef8b1e"
server_key_version: "encryptionkey@0"
I20260806 01:14:47.920543 6734 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260806 01:14:47.924436 6760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:47.925979 6734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:14:47.926333 6734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "aabd3ef1411f472c9ecac78f1dcbdf99"
format_stamp: "Formatted at 2026-08-06 01:14:47 on dist-test-slave-7nm7"
server_key: "b430b24b4926f0f3925cd595cdcb8c30"
server_key_iv: "94074bec3e8c4df3917d7772e7ef8b1e"
server_key_version: "encryptionkey@0"
I20260806 01:14:47.926566 6734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:47.943912 6734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:47.950587 6734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:47.950943 6734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:47.973795 6734 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:39649
I20260806 01:14:47.973820 6848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:39649 every 8 connection(s)
I20260806 01:14:47.975597 6734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:14:47.979988 6849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:14:47.981990 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 6734
I20260806 01:14:47.982246 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:14:47.982959 26546 external_mini_cluster.cc:1428] Setting key 9e1a9861630cdad9b876ffbfe7e1a61a
I20260806 01:14:47.994577 6849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99: Bootstrap starting.
I20260806 01:14:47.999411 6849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99: Neither blocks nor log segments found. Creating new log.
I20260806 01:14:48.000820 6849 log.cc:824] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99: Log is configured to *not* fsync() on all Append() calls
Aug 06 01:14:48 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:48.006157 6849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99: No bootstrap required, opened a new log
I20260806 01:14:48.016007 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:47.985585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41098 (local address 127.25.236.190:39649)
0806 01:14:47.986866 (+ 1281us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:47.986886 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:47.986976 (+ 90us) server_negotiation.cc:407] Connection header received
0806 01:14:47.989702 (+ 2726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:47.989733 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:47.990167 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:47.990659 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:47.992850 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:47.994501 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:47.996559 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:47.997305 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:47.997466 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:48.004418 (+ 6952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:48.004437 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:48.004458 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:48.004509 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:48.009538 (+ 5029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:48.010845 (+ 1307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:48.010855 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:48.010863 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:48.011100 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:48.011641 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:48.011645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:48.011647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:48.012356 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:48.012624 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:48.013595 (+ 971us) server_negotiation.cc:299] Negotiation successful
0806 01:14:48.014369 (+ 774us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":939,"thread_start_us":415,"threads_started":1}
I20260806 01:14:48.017050 6849 raft_consensus.cc:348] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39649 } }
I20260806 01:14:48.017596 6849 raft_consensus.cc:374] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:14:48.017756 6849 raft_consensus.cc:729] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aabd3ef1411f472c9ecac78f1dcbdf99, State: Initialized, Role: FOLLOWER
I20260806 01:14:48.018669 6849 consensus_queue.cc:260] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39649 } }
I20260806 01:14:48.019016 6849 raft_consensus.cc:388] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:14:48.019193 6849 raft_consensus.cc:482] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:14:48.019426 6849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:14:48.022534 6849 raft_consensus.cc:504] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39649 } }
I20260806 01:14:48.023459 6849 leader_election.cc:302] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aabd3ef1411f472c9ecac78f1dcbdf99; no voters:
I20260806 01:14:48.024646 6849 leader_election.cc:288] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:14:48.024744 6857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:14:48.026113 6857 raft_consensus.cc:686] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 1 LEADER]: Becoming Leader. State: Replica: aabd3ef1411f472c9ecac78f1dcbdf99, State: Running, Role: LEADER
I20260806 01:14:48.026746 6857 consensus_queue.cc:237] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39649 } }
I20260806 01:14:48.027392 6849 sys_catalog.cc:564] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:48.031595 6859 sys_catalog.cc:455] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aabd3ef1411f472c9ecac78f1dcbdf99. Latest consensus state: current_term: 1 leader_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39649 } } }
I20260806 01:14:48.032075 6859 sys_catalog.cc:458] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:48.032075 6861 sys_catalog.cc:455] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39649 } } }
I20260806 01:14:48.032430 6861 sys_catalog.cc:458] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:48.033986 6874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:48.040894 6874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:48.053458 6874 catalog_manager.cc:1347] Generated new cluster ID: d71056955c1449c5b20f2b8b17c1ec4c
I20260806 01:14:48.053663 6874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:48.072554 6874 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:14:48.074025 6874 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:48.087808 6874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99: Generated new TSK 0
I20260806 01:14:48.088744 6874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:14:48.161538 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39649
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:32937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:48.565814 6889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:48.566416 6889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:48.566721 6889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:48.574801 6889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:48.574918 6889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:48.575120 6889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:14:48.585042 6889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:32937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39649
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:48.586542 6889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:48.588032 6889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:48.603200 6900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:48.603106 6903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:48.603703 6901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:48.604636 6889 server_base.cc:1034] running on GCE node
I20260806 01:14:48.605278 6889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:48.606320 6889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:48.607586 6889 hybrid_clock.cc:648] HybridClock initialized: now 1785978888607531 us; error 43 us; skew 500 ppm
I20260806 01:14:48.607951 6889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:48 dist-test-slave-7nm7 krb5kdc[6711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978888, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:48.611495 6889 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:14:48.613150 6889 webserver.cc:460] Webserver started at http://127.25.236.129:41231/ using document root <none> and password file <none>
I20260806 01:14:48.614054 6889 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:48.614271 6889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:48.614610 6889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:48.617542 6889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "4de82bb1a3ae4a8684c8427148f3776e"
format_stamp: "Formatted at 2026-08-06 01:14:48 on dist-test-slave-7nm7"
server_key: "82f424af2d85fffd17c58953aa87c819"
server_key_iv: "5c032949e0ae584fea2f63624a8a9980"
server_key_version: "encryptionkey@0"
I20260806 01:14:48.618389 6889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "4de82bb1a3ae4a8684c8427148f3776e"
format_stamp: "Formatted at 2026-08-06 01:14:48 on dist-test-slave-7nm7"
server_key: "82f424af2d85fffd17c58953aa87c819"
server_key_iv: "5c032949e0ae584fea2f63624a8a9980"
server_key_version: "encryptionkey@0"
I20260806 01:14:48.624461 6889 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 01:14:48.628474 6913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:48.629841 6889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:14:48.630017 6889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "4de82bb1a3ae4a8684c8427148f3776e"
format_stamp: "Formatted at 2026-08-06 01:14:48 on dist-test-slave-7nm7"
server_key: "82f424af2d85fffd17c58953aa87c819"
server_key_iv: "5c032949e0ae584fea2f63624a8a9980"
server_key_version: "encryptionkey@0"
I20260806 01:14:48.630215 6889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:48.643893 6889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:48.648350 6889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:48.648638 6889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:48.649627 6889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:48.651086 6889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:48.651160 6889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:48.651301 6889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:48.651364 6889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:48.690040 6889 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:40259
I20260806 01:14:48.690339 7074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:40259 every 8 connection(s)
I20260806 01:14:48.691475 6889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:14:48.699745 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 6889
I20260806 01:14:48.699990 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:14:48.700917 26546 external_mini_cluster.cc:1428] Setting key a8de0e8507afd5d73defa37980ade233
Aug 06 01:14:48 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978888, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:48.716609 7078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:48.694417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:59947 (local address 127.25.236.190:39649)
0806 01:14:48.694874 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:48.694878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:48.696071 (+ 1193us) server_negotiation.cc:407] Connection header received
0806 01:14:48.697955 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:48.697961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:48.698050 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:48.698476 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:48.700153 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:48.700875 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:48.703906 (+ 3031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:48.704538 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:48.704897 (+ 359us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:48.708927 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:48.708938 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:48.708945 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:48.708994 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:48.713123 (+ 4129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:48.713924 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:48.713933 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:48.713940 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:48.714114 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:48.714667 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:48.714675 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:48.714681 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:48.715107 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:48.715276 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:48.716022 (+ 746us) server_negotiation.cc:299] Negotiation successful
0806 01:14:48.716301 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":244,"threads_started":1}
I20260806 01:14:48.717839 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:48.695072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:59947)
0806 01:14:48.695857 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:48.695913 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:48.697671 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:48.698719 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:48.698727 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:48.699307 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:48.699943 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:48.699961 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:48.701033 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:48.701039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:48.703764 (+ 2725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:48.703780 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:48.704987 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:48.704995 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:48.705295 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:48.706022 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:48.706046 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:48.708660 (+ 2614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:48.713321 (+ 4661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:48.713329 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:48.713343 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:48.713776 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:48.714316 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:48.714320 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:48.714322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:48.714517 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:48.715251 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:48.715257 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:48.715605 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 01:14:48.715842 (+ 237us) client_negotiation.cc:240] Negotiation successful
0806 01:14:48.716304 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":327,"threads_started":1}
I20260806 01:14:48.719521 7076 heartbeater.cc:347] Connected to a master server at 127.25.236.190:39649
I20260806 01:14:48.719877 7076 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:48.720634 7076 heartbeater.cc:511] Master 127.25.236.190:39649 requested a full tablet report, sending...
I20260806 01:14:48.722810 6793 ts_manager.cc:194] Registered new tserver with Master: 4de82bb1a3ae4a8684c8427148f3776e (127.25.236.129:40259)
I20260806 01:14:48.724242 6793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:59947
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:14:48.771885 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39649
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:32937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:49.129761 7084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:49.130182 7084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:49.130395 7084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:49.138649 7084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:49.138820 7084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:49.139038 7084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:14:49.149557 7084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:32937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39649
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:49.151059 7084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:49.152575 7084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:49.166658 7093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:49.166814 7094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:49.166846 7096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:49.168020 7084 server_base.cc:1034] running on GCE node
I20260806 01:14:49.168843 7084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:49.169991 7084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:49.171283 7084 hybrid_clock.cc:648] HybridClock initialized: now 1785978889171215 us; error 56 us; skew 500 ppm
I20260806 01:14:49.171662 7084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:49 dist-test-slave-7nm7 krb5kdc[6711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978889, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:49.175323 7084 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:14:49.177479 7084 webserver.cc:460] Webserver started at http://127.25.236.130:32947/ using document root <none> and password file <none>
I20260806 01:14:49.178328 7084 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:49.178503 7084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:49.178838 7084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:49.181915 7084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "3a249c7bbece49ccb29b0d31e4901343"
format_stamp: "Formatted at 2026-08-06 01:14:49 on dist-test-slave-7nm7"
server_key: "f359f5e5978d2e805fca3cb7dbad5894"
server_key_iv: "6d2afd89a45fac874ed1d572540cf9d7"
server_key_version: "encryptionkey@0"
I20260806 01:14:49.182813 7084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "3a249c7bbece49ccb29b0d31e4901343"
format_stamp: "Formatted at 2026-08-06 01:14:49 on dist-test-slave-7nm7"
server_key: "f359f5e5978d2e805fca3cb7dbad5894"
server_key_iv: "6d2afd89a45fac874ed1d572540cf9d7"
server_key_version: "encryptionkey@0"
I20260806 01:14:49.188973 7084 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260806 01:14:49.192816 7105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:49.194514 7084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:14:49.194761 7084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "3a249c7bbece49ccb29b0d31e4901343"
format_stamp: "Formatted at 2026-08-06 01:14:49 on dist-test-slave-7nm7"
server_key: "f359f5e5978d2e805fca3cb7dbad5894"
server_key_iv: "6d2afd89a45fac874ed1d572540cf9d7"
server_key_version: "encryptionkey@0"
I20260806 01:14:49.195014 7084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:49.212340 7084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:49.217075 7084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:49.217429 7084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:49.218580 7084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:49.220436 7084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:49.220558 7084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:49.220726 7084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:49.220817 7084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:49.261790 7084 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:43607
I20260806 01:14:49.261842 7281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:43607 every 8 connection(s)
I20260806 01:14:49.263216 7084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:14:49.264019 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 7084
I20260806 01:14:49.264220 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:14:49.264962 26546 external_mini_cluster.cc:1428] Setting key d973dfcfbda704aa75e0169df18772be
Aug 06 01:14:49 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978889, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:49.291771 7285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:49.266904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:49249 (local address 127.25.236.190:39649)
0806 01:14:49.267397 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:49.267401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:49.268552 (+ 1151us) server_negotiation.cc:407] Connection header received
0806 01:14:49.270473 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:49.270478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:49.270563 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:49.270714 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:49.273837 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:49.275101 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:49.278177 (+ 3076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:49.278766 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:49.278890 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:49.283928 (+ 5038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:49.283937 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:49.283944 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:49.283993 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:49.288065 (+ 4072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:49.288890 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:49.288898 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:49.288904 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:49.289066 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:49.289582 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:49.289589 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:49.289595 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:49.289992 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:49.290195 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:49.290988 (+ 793us) server_negotiation.cc:299] Negotiation successful
0806 01:14:49.291260 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":288,"threads_started":1}
I20260806 01:14:49.292287 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:49.267543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:49249)
0806 01:14:49.268384 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:49.268430 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:49.270133 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:49.271409 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:49.271430 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:49.272674 (+ 1244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:49.273578 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:49.273613 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:49.275345 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:49.275354 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:49.277843 (+ 2489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:49.277882 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:49.279131 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:49.279142 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:49.279506 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:49.280459 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:49.280494 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:49.283650 (+ 3156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:49.288254 (+ 4604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:49.288261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:49.288277 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:49.288750 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:49.289231 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:49.289235 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:49.289237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:49.289459 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:49.290148 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:49.290155 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:49.290601 (+ 446us) client_negotiation.cc:769] Sending connection context
0806 01:14:49.290854 (+ 253us) client_negotiation.cc:240] Negotiation successful
0806 01:14:49.291268 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":330,"threads_started":1}
I20260806 01:14:49.294010 7283 heartbeater.cc:347] Connected to a master server at 127.25.236.190:39649
I20260806 01:14:49.294399 7283 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:49.295375 7283 heartbeater.cc:511] Master 127.25.236.190:39649 requested a full tablet report, sending...
I20260806 01:14:49.297117 6793 ts_manager.cc:194] Registered new tserver with Master: 3a249c7bbece49ccb29b0d31e4901343 (127.25.236.130:43607)
I20260806 01:14:49.298625 6793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:49249
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:14:49.339160 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39649
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:32937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:49.690786 7298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:49.691053 7298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:49.691212 7298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:49.699011 7298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:49.699131 7298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:49.699316 7298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:14:49.711292 7298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:32937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39649
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:49.712705 7298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:49.714074 7298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 01:14:49.727581 7076 heartbeater.cc:503] Master 127.25.236.190:39649 was elected leader, sending a full tablet report...
W20260806 01:14:49.730278 7304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:49.730288 7309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:49.730432 7305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:49.731374 7298 server_base.cc:1034] running on GCE node
I20260806 01:14:49.732064 7298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:49.733111 7298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:49.734352 7298 hybrid_clock.cc:648] HybridClock initialized: now 1785978889734291 us; error 42 us; skew 500 ppm
I20260806 01:14:49.734740 7298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:49 dist-test-slave-7nm7 krb5kdc[6711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978889, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:49.738385 7298 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:14:49.740161 7298 webserver.cc:460] Webserver started at http://127.25.236.131:45191/ using document root <none> and password file <none>
I20260806 01:14:49.740974 7298 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:49.741220 7298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:49.741892 7298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:14:49.745070 7298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "a220eebc14904ba5b2435c4e09c33c30"
format_stamp: "Formatted at 2026-08-06 01:14:49 on dist-test-slave-7nm7"
server_key: "b4c03a37c10efd6e35f8a739136fcc21"
server_key_iv: "f378f674d0e6efb861687caac38950d2"
server_key_version: "encryptionkey@0"
I20260806 01:14:49.745904 7298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "a220eebc14904ba5b2435c4e09c33c30"
format_stamp: "Formatted at 2026-08-06 01:14:49 on dist-test-slave-7nm7"
server_key: "b4c03a37c10efd6e35f8a739136fcc21"
server_key_iv: "f378f674d0e6efb861687caac38950d2"
server_key_version: "encryptionkey@0"
I20260806 01:14:49.751291 7298 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:14:49.754856 7323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:49.756248 7298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 01:14:49.756439 7298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "a220eebc14904ba5b2435c4e09c33c30"
format_stamp: "Formatted at 2026-08-06 01:14:49 on dist-test-slave-7nm7"
server_key: "b4c03a37c10efd6e35f8a739136fcc21"
server_key_iv: "f378f674d0e6efb861687caac38950d2"
server_key_version: "encryptionkey@0"
I20260806 01:14:49.756650 7298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:49.775071 7298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:49.779961 7298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:49.780398 7298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:49.782300 7298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:49.784372 7298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:49.784507 7298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:49.784658 7298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:49.784746 7298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:49.830868 7298 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:34707
I20260806 01:14:49.830996 7489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:34707 every 8 connection(s)
I20260806 01:14:49.832293 7298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:14:49.841668 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 7298
I20260806 01:14:49.841879 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:14:49.842518 26546 external_mini_cluster.cc:1428] Setting key 9eea101deb24d7441fd28d133945e60b
Aug 06 01:14:49 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978889, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:14:49.859393 7495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:49.835657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:48871 (local address 127.25.236.190:39649)
0806 01:14:49.836280 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:49.836287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:49.837599 (+ 1312us) server_negotiation.cc:407] Connection header received
0806 01:14:49.839549 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:49.839554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:49.839649 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:49.839758 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:49.841619 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:49.842869 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:49.846017 (+ 3148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:49.846704 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:49.846871 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:49.851166 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:49.851172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:49.851176 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:49.851213 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:49.854129 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:49.855095 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:49.855100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:49.855104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:49.855429 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:49.855972 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:49.856119 (+ 147us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:49.856134 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:49.856690 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:49.856929 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:49.858461 (+ 1532us) server_negotiation.cc:299] Negotiation successful
0806 01:14:49.858879 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":272,"threads_started":1}
I20260806 01:14:49.860771 7496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:49.836489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:48871)
0806 01:14:49.837383 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:49.837446 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:49.839266 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:49.840027 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:49.840036 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:49.840790 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:49.841416 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:49.841434 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:49.843204 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:49.843211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:49.845797 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:49.845819 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:49.847076 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:49.847084 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:49.847390 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:49.848289 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:49.848318 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:49.850888 (+ 2570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:14:49.854419 (+ 3531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:49.854429 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:49.854444 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:49.854922 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:49.855586 (+ 664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:14:49.855591 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:14:49.855595 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:14:49.855808 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:14:49.856917 (+ 1109us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:14:49.856923 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:14:49.857797 (+ 874us) client_negotiation.cc:769] Sending connection context
0806 01:14:49.858204 (+ 407us) client_negotiation.cc:240] Negotiation successful
0806 01:14:49.859190 (+ 986us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":382,"threads_started":1}
I20260806 01:14:49.862913 7493 heartbeater.cc:347] Connected to a master server at 127.25.236.190:39649
I20260806 01:14:49.863279 7493 heartbeater.cc:464] Registering TS with master...
I20260806 01:14:49.864079 7493 heartbeater.cc:511] Master 127.25.236.190:39649 requested a full tablet report, sending...
I20260806 01:14:49.865834 6793 ts_manager.cc:194] Registered new tserver with Master: a220eebc14904ba5b2435c4e09c33c30 (127.25.236.131:34707)
I20260806 01:14:49.866776 6793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:48871
I20260806 01:14:49.874506 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab.
Entry for principal oryx/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab.
WARNING: no policy specified for oryx/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab.
Entry for principal oryx/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab.
WARNING: no policy specified for oryx/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab.
Entry for principal oryx/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab.
I20260806 01:14:50.073484 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 6889
Aug 06 01:14:50 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:14:50.092005 7514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:50.074652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41222 (local address 127.25.236.129:40259)
0806 01:14:50.075338 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:50.075345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:50.075377 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:14:50.075444 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:50.075451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:50.075672 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:50.075815 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:50.078050 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:50.079450 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:50.080729 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:50.081352 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:50.081460 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:50.085639 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:50.085651 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:50.085666 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:50.085704 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:50.088892 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:50.089679 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:50.089700 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:50.089702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:50.089848 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:50.090371 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:50.090375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:50.090377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:50.090744 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:50.091091 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:50.091357 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 01:14:50.091582 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":331,"threads_started":1}
I20260806 01:14:50.093392 7067 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:50.183275 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 6889
I20260806 01:14:50.212459 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 7084
Aug 06 01:14:50 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:14:50.230975 7516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:50.213503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50532 (local address 127.25.236.130:43607)
0806 01:14:50.214052 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:50.214060 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:50.214108 (+ 48us) server_negotiation.cc:407] Connection header received
0806 01:14:50.214174 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:50.214181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:50.214462 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:50.214597 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:50.215845 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:50.217000 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:50.218427 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:50.219041 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:50.219172 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:50.224563 (+ 5391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:50.224576 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:50.224591 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:50.224629 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:50.227960 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:50.228729 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:50.228745 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:50.228747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:50.228880 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:50.229261 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:50.229266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:50.229269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:50.229614 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:50.229940 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:50.230300 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 01:14:50.230622 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":292,"threads_started":1}
I20260806 01:14:50.232766 7276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:50.325284 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 7084
I20260806 01:14:50.354460 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 7298
Aug 06 01:14:50 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:14:50.374634 7520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:50.355622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38780 (local address 127.25.236.131:34707)
0806 01:14:50.356465 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:50.356475 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:50.356524 (+ 49us) server_negotiation.cc:407] Connection header received
0806 01:14:50.356617 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:50.356625 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:50.357035 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:50.357469 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:50.359516 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:50.360818 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:50.363113 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:50.363848 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:50.363967 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:50.368812 (+ 4845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:50.368824 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:50.368840 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:50.368875 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:50.371983 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:50.372634 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:50.372641 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:50.372644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:50.372762 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:50.373076 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:50.373079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:50.373081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:50.373440 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:50.373677 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:50.373961 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 01:14:50.374213 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":378,"threads_started":1}
I20260806 01:14:50.376436 7482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:50.465034 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 7298
I20260806 01:14:50.490171 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 6734
I20260806 01:14:50.491519 6840 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:14:50.599041 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 6734
I20260806 01:14:50.630537 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:39649
--webserver_interface=127.25.236.190
--webserver_port=46855
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:32937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:39649
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:14:50.986824 7523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:50.987200 7523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:50.987368 7523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:50.995254 7523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:14:50.995432 7523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:50.995563 7523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:50.995674 7523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:14:50.995769 7523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:14:51.006536 7523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:32937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:39649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:39649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=46855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:51.008013 7523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:51.009377 7523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:51.022747 7532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:51.022812 7533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:51.022840 7535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:51.024857 7523 server_base.cc:1034] running on GCE node
I20260806 01:14:51.025812 7523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:51.027539 7523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:51.028784 7523 hybrid_clock.cc:648] HybridClock initialized: now 1785978891028739 us; error 38 us; skew 500 ppm
I20260806 01:14:51.029233 7523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:51 dist-test-slave-7nm7 krb5kdc[6711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978891, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:51.037231 7523 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:14:51.039119 7523 webserver.cc:460] Webserver started at http://127.25.236.190:46855/ using document root <none> and password file <none>
I20260806 01:14:51.039881 7523 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:51.040048 7523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:51.046891 7523 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:14:51.050199 7548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:51.051767 7523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:14:51.051915 7523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "aabd3ef1411f472c9ecac78f1dcbdf99"
format_stamp: "Formatted at 2026-08-06 01:14:47 on dist-test-slave-7nm7"
server_key: "b430b24b4926f0f3925cd595cdcb8c30"
server_key_iv: "94074bec3e8c4df3917d7772e7ef8b1e"
server_key_version: "encryptionkey@0"
I20260806 01:14:51.052917 7523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:51.085264 7523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:51.090394 7523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:51.090689 7523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:51.113040 7523 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:39649
I20260806 01:14:51.113059 7627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:39649 every 8 connection(s)
I20260806 01:14:51.114820 7523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:14:51.121865 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 7523
I20260806 01:14:51.121896 7628 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:14:51.122803 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:40259
--local_ip_for_outbound_sockets=127.25.236.129
--tserver_master_addrs=127.25.236.190:39649
--webserver_port=41231
--webserver_interface=127.25.236.129
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:32937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.236.129
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:14:51.125376 7628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99: Bootstrap starting.
I20260806 01:14:51.149287 7628 log.cc:824] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99: Log is configured to *not* fsync() on all Append() calls
I20260806 01:14:51.162911 7628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:14:51.163645 7628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99: Bootstrap complete.
I20260806 01:14:51.178731 7628 raft_consensus.cc:348] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39649 } }
I20260806 01:14:51.180154 7628 raft_consensus.cc:729] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aabd3ef1411f472c9ecac78f1dcbdf99, State: Initialized, Role: FOLLOWER
I20260806 01:14:51.180927 7628 consensus_queue.cc:260] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39649 } }
I20260806 01:14:51.181221 7628 raft_consensus.cc:388] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:14:51.181348 7628 raft_consensus.cc:482] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:14:51.181532 7628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:14:51.187355 7628 raft_consensus.cc:504] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39649 } }
I20260806 01:14:51.187983 7628 leader_election.cc:302] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aabd3ef1411f472c9ecac78f1dcbdf99; no voters:
I20260806 01:14:51.188920 7628 leader_election.cc:288] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 01:14:51.189055 7633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:14:51.190728 7633 raft_consensus.cc:686] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [term 2 LEADER]: Becoming Leader. State: Replica: aabd3ef1411f472c9ecac78f1dcbdf99, State: Running, Role: LEADER
I20260806 01:14:51.191334 7633 consensus_queue.cc:237] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39649 } }
I20260806 01:14:51.191418 7628 sys_catalog.cc:564] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:14:51.193674 7636 sys_catalog.cc:455] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aabd3ef1411f472c9ecac78f1dcbdf99. Latest consensus state: current_term: 2 leader_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39649 } } }
I20260806 01:14:51.194106 7636 sys_catalog.cc:458] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:51.194764 7635 sys_catalog.cc:455] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aabd3ef1411f472c9ecac78f1dcbdf99" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39649 } } }
I20260806 01:14:51.195123 7635 sys_catalog.cc:458] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99 [sys.catalog]: This master's current role is: LEADER
I20260806 01:14:51.200346 7640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:14:51.208863 7640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:14:51.210721 7640 catalog_manager.cc:1259] Loaded cluster ID: d71056955c1449c5b20f2b8b17c1ec4c
I20260806 01:14:51.210840 7640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:14:51.216516 7640 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:14:51.218478 7640 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aabd3ef1411f472c9ecac78f1dcbdf99: Loaded TSK: 0
I20260806 01:14:51.220350 7640 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 01:14:51.536751 7630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:51.537155 7630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:51.537364 7630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:51.545382 7630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:51.545549 7630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:51.545753 7630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:14:51.556454 7630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:32937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.129.keytab
--principal=oryx/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:40259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=41231
--tserver_master_addrs=127.25.236.190:39649
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:51.557914 7630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:51.559330 7630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:51.573637 7672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:51.573606 7671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:51.574561 7630 server_base.cc:1034] running on GCE node
W20260806 01:14:51.574993 7675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:51.575781 7630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:51.576880 7630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:51.578111 7630 hybrid_clock.cc:648] HybridClock initialized: now 1785978891578050 us; error 52 us; skew 500 ppm
I20260806 01:14:51.578480 7630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:51 dist-test-slave-7nm7 krb5kdc[6711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978891, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:51.582173 7630 init.cc:377] Logged in from keytab as oryx/127.25.236.129@KRBTEST.COM (short username oryx)
I20260806 01:14:51.583925 7630 webserver.cc:460] Webserver started at http://127.25.236.129:41231/ using document root <none> and password file <none>
I20260806 01:14:51.584604 7630 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:51.584756 7630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:51.590447 7630 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:14:51.593600 7684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:51.595456 7630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:14:51.595672 7630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "4de82bb1a3ae4a8684c8427148f3776e"
format_stamp: "Formatted at 2026-08-06 01:14:48 on dist-test-slave-7nm7"
server_key: "82f424af2d85fffd17c58953aa87c819"
server_key_iv: "5c032949e0ae584fea2f63624a8a9980"
server_key_version: "encryptionkey@0"
I20260806 01:14:51.596866 7630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:51.625386 7630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:51.630038 7630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:51.630393 7630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:51.631493 7630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:51.633183 7630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:51.633334 7630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:51.633478 7630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:51.633571 7630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:51.669991 7630 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:40259
I20260806 01:14:51.670032 7871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:40259 every 8 connection(s)
I20260806 01:14:51.671591 7630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:14:51.680696 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 7630
I20260806 01:14:51.681347 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:43607
--local_ip_for_outbound_sockets=127.25.236.130
--tserver_master_addrs=127.25.236.190:39649
--webserver_port=32947
--webserver_interface=127.25.236.130
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:32937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.236.130
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:14:51 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:51 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:14:51.691130 7875 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:51.674670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:42313 (local address 127.25.236.190:39649)
0806 01:14:51.675453 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:51.675469 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:51.675784 (+ 315us) server_negotiation.cc:407] Connection header received
0806 01:14:51.678081 (+ 2297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:51.678093 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:51.678458 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:51.678798 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:51.680290 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:51.681620 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:51.683170 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:51.683775 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:51.683983 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:51.690122 (+ 6139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:42313: BlockingRecv error: recv got EOF from 127.25.236.129:42313 (error 108)
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":239,"threads_started":1}
I20260806 01:14:51.691148 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:51.674788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:42313)
0806 01:14:51.675604 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:51.675658 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:51.677527 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:51.678937 (+ 1410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:51.678949 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:51.679558 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:51.680079 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:51.680095 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:51.681770 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:51.681780 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:51.683014 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:51.683026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:51.684018 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:51.684026 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:51.684290 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:51.684987 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:51.685009 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:51.689756 (+ 4747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":368,"threads_started":1}
W20260806 01:14:51.691928 7872 heartbeater.cc:644] Failed to heartbeat to 127.25.236.190:39649 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.236.190:39649: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Aug 06 01:14:51 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:51 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:51.703212 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:51.692726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:52557)
0806 01:14:51.692930 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:51.692972 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:51.693089 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:51.693481 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:51.693486 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:51.693840 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:51.693984 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:51.693992 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:51.695274 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:51.695282 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:51.696595 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:51.696608 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:51.697353 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:51.697359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:51.697532 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:51.698354 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:51.698373 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:51.702806 (+ 4433us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 01:14:51.703423 7875 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:51.692898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:52557 (local address 127.25.236.190:39649)
0806 01:14:51.693087 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:51.693093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:51.693112 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:14:51.693270 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:51.693283 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:51.693343 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:51.693455 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:51.694142 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:51.695025 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:51.696771 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:51.697187 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:51.697326 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:51.703042 (+ 5716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:52557: BlockingRecv error: recv got EOF from 127.25.236.129:52557 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 01:14:51 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:51 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:14:51.713900 7875 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:51.704558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:55813 (local address 127.25.236.190:39649)
0806 01:14:51.704757 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:51.704763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:51.704782 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:14:51.704946 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:51.704951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:51.705040 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:51.705156 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:51.705829 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:51.706672 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:51.708122 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:51.708537 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:51.708689 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:51.713713 (+ 5024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:55813: BlockingRecv error: recv got EOF from 127.25.236.129:55813 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:14:51.713894 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:51.704423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:55813)
0806 01:14:51.704638 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:51.704658 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:51.704789 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:51.705185 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:51.705190 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:51.705516 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:51.705679 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:51.705688 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:51.706822 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:51.706828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:51.707970 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:51.707984 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:51.708699 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:51.708704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:51.708885 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:51.709565 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:51.709585 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:51.713489 (+ 3904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 01:14:51.714427 7872 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:14:52.032363 7877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:52.032660 7877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:52.032819 7877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:52.040223 7877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:52.040383 7877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:52.040532 7877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:14:52.050464 7877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:32937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.130.keytab
--principal=oryx/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:43607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=32947
--tserver_master_addrs=127.25.236.190:39649
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:52.051882 7877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:52.053383 7877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:52.069368 7888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:52.069401 7887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:52.069445 7891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:52.070393 7877 server_base.cc:1034] running on GCE node
I20260806 01:14:52.071025 7877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:52.072085 7877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:52.073310 7877 hybrid_clock.cc:648] HybridClock initialized: now 1785978892073263 us; error 44 us; skew 500 ppm
I20260806 01:14:52.073616 7877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978892, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:52.082598 7877 init.cc:377] Logged in from keytab as oryx/127.25.236.130@KRBTEST.COM (short username oryx)
I20260806 01:14:52.084599 7877 webserver.cc:460] Webserver started at http://127.25.236.130:32947/ using document root <none> and password file <none>
I20260806 01:14:52.085357 7877 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:52.085521 7877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:52.092051 7877 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:14:52.095098 7903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:52.096725 7877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:14:52.096932 7877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "3a249c7bbece49ccb29b0d31e4901343"
format_stamp: "Formatted at 2026-08-06 01:14:49 on dist-test-slave-7nm7"
server_key: "f359f5e5978d2e805fca3cb7dbad5894"
server_key_iv: "6d2afd89a45fac874ed1d572540cf9d7"
server_key_version: "encryptionkey@0"
I20260806 01:14:52.097937 7877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:52.127877 7877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:52.132140 7877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:52.132447 7877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:52.133452 7877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:52.134877 7877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:52.135002 7877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:52.135138 7877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:52.135229 7877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:52.169831 7877 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:43607
I20260806 01:14:52.169919 8075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:43607 every 8 connection(s)
I20260806 01:14:52.171327 7877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:14:52.177011 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 7877
I20260806 01:14:52.177745 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:34707
--local_ip_for_outbound_sockets=127.25.236.131
--tserver_master_addrs=127.25.236.190:39649
--webserver_port=45191
--webserver_interface=127.25.236.131
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:32937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.236.131
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:14:52.187881 7875 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:52.174071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:45783 (local address 127.25.236.190:39649)
0806 01:14:52.174315 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:52.174320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:52.175403 (+ 1083us) server_negotiation.cc:407] Connection header received
0806 01:14:52.177339 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:52.177345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:52.177437 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:52.177552 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:52.179183 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.179878 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.181479 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.181917 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.182048 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.187680 (+ 5632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:45783: BlockingRecv error: recv got EOF from 127.25.236.130:45783 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:14:52.188817 8079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:52.174558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:45783)
0806 01:14:52.175250 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:52.175286 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:52.176971 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:52.177788 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:52.177796 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:52.178476 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:52.179000 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.179017 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.180046 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.180050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.181323 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.181346 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.182166 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.182175 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.182452 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.183128 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:52.183157 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:52.187517 (+ 4360us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":277,"threads_started":1}
W20260806 01:14:52.189592 8076 heartbeater.cc:644] Failed to heartbeat to 127.25.236.190:39649 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.236.190:39649: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:14:52.200731 7875 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:52.190339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:35993 (local address 127.25.236.190:39649)
0806 01:14:52.190532 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:52.190536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:52.190553 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:14:52.190689 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:52.190697 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:52.190781 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:52.190890 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:52.191557 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.192775 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.194045 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.194957 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.195528 (+ 571us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.200540 (+ 5012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:35993: BlockingRecv error: recv got EOF from 127.25.236.130:35993 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 01:14:52.200742 8079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:52.190289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:35993)
0806 01:14:52.190464 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:52.190489 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:52.190569 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:52.190918 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:52.190921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:52.191278 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:52.191414 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.191422 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.192920 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.192924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.193899 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.193911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.195352 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.195357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.195561 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.196376 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:52.196395 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:52.200259 (+ 3864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:52.210171 8079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:52.201615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:46739)
0806 01:14:52.201886 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:52.201900 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:52.202019 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:52.202355 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:52.202360 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:52.202672 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:52.202837 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.202846 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.204038 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.204043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.205013 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.205025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.205870 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.205876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.206061 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.206724 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:52.206741 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:52.209868 (+ 3127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160}
W20260806 01:14:52.210358 7875 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:52.201886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:46739 (local address 127.25.236.190:39649)
0806 01:14:52.202050 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:52.202058 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:52.202079 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:14:52.202134 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:52.202140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:52.202204 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:52.202332 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:52.202976 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.203890 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.205167 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.205695 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.205860 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.210083 (+ 4223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:46739: BlockingRecv error: recv got EOF from 127.25.236.130:46739 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 01:14:52.210726 8076 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:14:52.566455 8081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:14:52.566792 8081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:14:52.566962 8081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:14:52.574431 8081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:14:52.574579 8081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:14:52.574760 8081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:14:52.584497 8081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:32937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/krb5kdc/oryx_127.25.236.131.keytab
--principal=oryx/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:34707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=45191
--tserver_master_addrs=127.25.236.190:39649
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:14:52.585913 8081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:14:52.587307 8081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:14:52.601120 8090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:52.601043 8095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:14:52.601089 8091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:14:52.604125 8081 server_base.cc:1034] running on GCE node
I20260806 01:14:52.604902 8081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:14:52.606003 8081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:14:52.607218 8081 hybrid_clock.cc:648] HybridClock initialized: now 1785978892607168 us; error 35 us; skew 500 ppm
I20260806 01:14:52.607563 8081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978892, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:14:52.610800 8081 init.cc:377] Logged in from keytab as oryx/127.25.236.131@KRBTEST.COM (short username oryx)
I20260806 01:14:52.612397 8081 webserver.cc:460] Webserver started at http://127.25.236.131:45191/ using document root <none> and password file <none>
I20260806 01:14:52.613111 8081 fs_manager.cc:359] Metadata directory not provided
I20260806 01:14:52.613255 8081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:14:52.618667 8081 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260806 01:14:52.621291 8112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:52.622778 8081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:14:52.622956 8081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "a220eebc14904ba5b2435c4e09c33c30"
format_stamp: "Formatted at 2026-08-06 01:14:49 on dist-test-slave-7nm7"
server_key: "b4c03a37c10efd6e35f8a739136fcc21"
server_key_iv: "f378f674d0e6efb861687caac38950d2"
server_key_version: "encryptionkey@0"
I20260806 01:14:52.623926 8081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:14:52.642802 8081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:14:52.647560 8081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:14:52.647866 8081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:14:52.648922 8081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:14:52.650377 8081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:14:52.650501 8081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:52.650638 8081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:14:52.650730 8081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:14:52.689661 8081 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:34707
I20260806 01:14:52.689726 8268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:34707 every 8 connection(s)
I20260806 01:14:52.691315 8081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:14:52.694543 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 8081
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:14:52.709514 7875 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:52.693894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:41107 (local address 127.25.236.190:39649)
0806 01:14:52.694111 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:52.694118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:52.695614 (+ 1496us) server_negotiation.cc:407] Connection header received
0806 01:14:52.698443 (+ 2829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:52.698447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:52.698519 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:52.698600 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:52.700255 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.701135 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.703484 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.703941 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.704034 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.709215 (+ 5181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:41107: BlockingRecv error: recv got EOF from 127.25.236.131:41107 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:14:52.710186 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:52.694532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:41107)
0806 01:14:52.695444 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:52.695506 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:52.698108 (+ 2602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:52.698891 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:52.698905 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:52.699450 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:52.699988 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.700004 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.701346 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.701352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.703302 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.703315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.704190 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.704198 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.704535 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.705186 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:52.705219 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:52.708982 (+ 3763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":351,"threads_started":1}
W20260806 01:14:52.710969 8269 heartbeater.cc:644] Failed to heartbeat to 127.25.236.190:39649 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.236.190:39649: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
I20260806 01:14:52.712597 8274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:52.695667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41110 (local address 127.25.236.190:39649)
0806 01:14:52.696239 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:52.696246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:52.696569 (+ 323us) server_negotiation.cc:407] Connection header received
0806 01:14:52.696767 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:52.696771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:52.696834 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:52.696925 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:52.698998 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.700239 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.701962 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.702505 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.702600 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.705885 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:14:52.705893 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:14:52.705905 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:14:52.705937 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:14:52.709018 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:52.709810 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:52.709816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:52.709822 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:52.709981 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:14:52.710440 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:14:52.710445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:14:52.710449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:14:52.711061 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:14:52.711399 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 01:14:52.711818 (+ 419us) server_negotiation.cc:299] Negotiation successful
0806 01:14:52.712203 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":284,"threads_started":1}
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:52.720851 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:52.711634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:56447)
0806 01:14:52.711891 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:52.711909 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:52.712018 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:52.712361 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:52.712364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:52.712676 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:52.712866 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.712875 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.714171 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.714177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.715449 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.715464 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.716692 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.716696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.716880 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.717532 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:52.717582 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:52.720648 (+ 3066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132}
W20260806 01:14:52.721239 7875 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:52.711728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:56447 (local address 127.25.236.190:39649)
0806 01:14:52.711883 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:52.711889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:52.711962 (+ 73us) server_negotiation.cc:407] Connection header received
0806 01:14:52.712136 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:52.712142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:52.712210 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:52.712314 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:52.713008 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.714021 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.715625 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.716353 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.716450 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.720912 (+ 4462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:56447: BlockingRecv error: recv got EOF from 127.25.236.131:56447 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:52.731449 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:52.721582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:58443)
0806 01:14:52.721986 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:52.722010 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:52.722143 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:52.722520 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:52.722527 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:52.723039 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:52.723283 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.723297 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.724296 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.724304 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.726305 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.726324 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.727347 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.727377 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.727609 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.728212 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:52.728234 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:52.731252 (+ 3018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 01:14:52.731791 8269 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:14:52.732157 7875 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:52.721659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:58443 (local address 127.25.236.190:39649)
0806 01:14:52.721832 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:52.721840 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:52.722098 (+ 258us) server_negotiation.cc:407] Connection header received
0806 01:14:52.722319 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:52.722323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:52.722377 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:52.722455 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:52.723452 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.724159 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.726481 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.727077 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.727219 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.731499 (+ 4280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:58443: BlockingRecv error: recv got EOF from 127.25.236.131:58443 (error 108)
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":35}
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:52 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:52.735714 8276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:52.714978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:55355)
0806 01:14:52.726714 (+ 11736us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:52.726739 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:52.726879 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:52.727207 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:52.727212 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:52.727925 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:52.728202 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.728216 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.729636 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.729639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.730854 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:52.730870 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.731901 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.731910 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:52.732090 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.732651 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:52.732667 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:52.735507 (+ 2840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":11608,"thread_start_us":11474,"threads_started":1}
W20260806 01:14:52.736006 8274 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:52.715413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:55355 (local address 127.25.236.190:39649)
0806 01:14:52.715679 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:52.715687 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:52.726832 (+ 11145us) server_negotiation.cc:407] Connection header received
0806 01:14:52.726998 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:52.727001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:52.727051 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:52.727166 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:52.728383 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.729480 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.730996 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:52.731525 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:52.731656 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:52.735697 (+ 4041us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:55355: BlockingRecv error: recv got EOF from 127.25.236.129:55355 (error 108)
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":139}
Aug 06 01:14:53 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:53 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:53.222347 8279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:53.211576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:47019)
0806 01:14:53.212175 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:53.212192 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:53.212397 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:53.212869 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:53.212873 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:53.213697 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:53.213913 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:53.213923 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:53.215502 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:53.215523 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:53.216704 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:53.216721 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:53.217769 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:53.217776 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:53.217972 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:53.218860 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:53.218884 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:53.221977 (+ 3093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":308,"threads_started":1}
W20260806 01:14:53.222491 8274 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:53.211690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:47019 (local address 127.25.236.190:39649)
0806 01:14:53.211954 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:53.211963 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:53.212402 (+ 439us) server_negotiation.cc:407] Connection header received
0806 01:14:53.212640 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:53.212644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:53.212711 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:53.212825 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:53.214076 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:53.215008 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:53.217061 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:53.217565 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:53.217715 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:53.222188 (+ 4473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:47019: BlockingRecv error: recv got EOF from 127.25.236.130:47019 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 06 01:14:53 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:53 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:53.742319 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:53.732454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:54125)
0806 01:14:53.732986 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:53.733004 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:53.733117 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:53.733415 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:53.733418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:53.733895 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:53.734084 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:53.734093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:53.735208 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:53.735212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:53.736148 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:53.736160 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:53.737259 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:53.737268 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:53.737536 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:53.738376 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:53.738401 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:53.741928 (+ 3527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":292,"threads_started":1}
W20260806 01:14:53.742449 8283 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:53.732550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:54125 (local address 127.25.236.190:39649)
0806 01:14:53.733104 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:53.733108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:53.733127 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:14:53.733208 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:53.733212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:53.733279 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:53.733378 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:53.734251 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:53.735071 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:53.736315 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:53.737006 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:53.737192 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:53.742136 (+ 4944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:54125: BlockingRecv error: recv got EOF from 127.25.236.131:54125 (error 108)
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":324,"threads_started":1}
Aug 06 01:14:53 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:53 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:53.746407 8284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:53.736625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:48635)
0806 01:14:53.738071 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:53.738102 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:53.738281 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:53.738650 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:53.738653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:53.739139 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:53.739402 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:53.739414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:53.740905 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:53.740910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:53.741935 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:53.741947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:53.742887 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:53.742893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:53.743069 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:53.743532 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:53.743547 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:53.746066 (+ 2519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1286,"thread_start_us":1161,"threads_started":1}
W20260806 01:14:53.746829 8287 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:53.736699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:48635 (local address 127.25.236.190:39649)
0806 01:14:53.737796 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:53.737804 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:53.738188 (+ 384us) server_negotiation.cc:407] Connection header received
0806 01:14:53.738415 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:53.738419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:53.738493 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:53.738608 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:53.739583 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:53.740770 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:53.742091 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:53.742742 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:53.742865 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:53.746430 (+ 3565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:48635: BlockingRecv error: recv got EOF from 127.25.236.129:48635 (error 108)
Metrics: {"server-negotiator.queue_time_us":964,"thread_start_us":844,"threads_started":1}
Aug 06 01:14:54 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:54 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:54.232892 8288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:54.223579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:43047)
0806 01:14:54.224199 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:54.224216 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:54.224393 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:54.224761 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:54.224766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:54.225322 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:54.225500 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:54.225509 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:54.226628 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:54.226634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:54.227855 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:54.227868 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:54.228775 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:54.228781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:54.228949 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:54.229642 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:54.229662 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:54.232525 (+ 2863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":348,"threads_started":1}
W20260806 01:14:54.232975 8287 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:54.223714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:43047 (local address 127.25.236.190:39649)
0806 01:14:54.223998 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:54.224006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:54.224325 (+ 319us) server_negotiation.cc:407] Connection header received
0806 01:14:54.224542 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:54.224546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:54.224607 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:54.224714 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:54.225650 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:54.226484 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:54.228012 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:54.228592 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:54.228746 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:54.232737 (+ 3991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:43047: BlockingRecv error: recv got EOF from 127.25.236.130:43047 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 06 01:14:54 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:54 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:54.754745 8289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:54.743318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:44055)
0806 01:14:54.743875 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:54.743893 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:54.744001 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:54.744440 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:54.744444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:54.744869 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:54.745066 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:54.745075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:54.746374 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:54.746381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:54.747965 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:54.747990 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:54.748899 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:54.748904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:54.749035 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:54.749976 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:54.750001 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:54.754358 (+ 4357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":307,"threads_started":1}
W20260806 01:14:54.754904 8290 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:54.743631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:44055 (local address 127.25.236.190:39649)
0806 01:14:54.744138 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:54.744142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:54.744158 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:14:54.744204 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:54.744207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:54.744277 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:54.744367 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:54.745255 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:54.746051 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:54.748268 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:54.748750 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:54.748850 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:54.754600 (+ 5750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:44055: BlockingRecv error: recv got EOF from 127.25.236.131:44055 (error 108)
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":283,"threads_started":1}
Aug 06 01:14:54 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:54 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:14:54.758594 8292 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:54.747598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:48759 (local address 127.25.236.190:39649)
0806 01:14:54.748380 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:54.748386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:54.749078 (+ 692us) server_negotiation.cc:407] Connection header received
0806 01:14:54.749271 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:54.749276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:54.749342 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:54.749450 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:54.750208 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:54.750998 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:54.752435 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:54.753122 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:54.753249 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:54.758383 (+ 5134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:48759: BlockingRecv error: recv got EOF from 127.25.236.129:48759 (error 108)
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":565,"threads_started":1}
I20260806 01:14:54.758666 8291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:54.747520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:48759)
0806 01:14:54.748061 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:54.748225 (+ 164us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:54.749150 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:54.749486 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:54.749494 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:54.749871 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:54.750049 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:54.750058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:54.751143 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:54.751149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:54.752217 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:54.752235 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:54.753330 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:54.753337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:54.753513 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:54.754483 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:54.754506 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:54.758115 (+ 3609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":284,"threads_started":1}
Aug 06 01:14:55 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:55 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:55.242980 8293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:55.233992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:47427)
0806 01:14:55.234618 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:55.234636 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:55.234754 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:55.235062 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:55.235065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:55.235438 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:55.235619 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:55.235628 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:55.236708 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:55.236712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:55.237607 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:55.237619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:55.238381 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:55.238386 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:55.238523 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:55.239270 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:55.239288 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:55.242577 (+ 3289us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":399,"threads_started":1}
W20260806 01:14:55.243045 8292 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:55.234044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:47427 (local address 127.25.236.190:39649)
0806 01:14:55.234405 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:55.234412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:55.234724 (+ 312us) server_negotiation.cc:407] Connection header received
0806 01:14:55.234861 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:55.234865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:55.234928 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:55.235015 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:55.235791 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:55.236555 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:55.237751 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:55.238185 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:55.238311 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:55.242795 (+ 4484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:47427: BlockingRecv error: recv got EOF from 127.25.236.130:47427 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
Aug 06 01:14:55 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:55 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:55.766916 8294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:55.756101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:56925)
0806 01:14:55.756731 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:55.756747 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:55.756860 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:55.757150 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:55.757153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:55.757678 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:55.757901 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:55.757913 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:55.759276 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:55.759282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:55.760790 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:55.760807 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:55.761766 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:55.761773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:55.762023 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:55.762773 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:55.762802 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:55.766518 (+ 3716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":319,"threads_started":1}
W20260806 01:14:55.766965 8295 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:55.756222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:56925 (local address 127.25.236.190:39649)
0806 01:14:55.756691 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:55.756695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:55.756820 (+ 125us) server_negotiation.cc:407] Connection header received
0806 01:14:55.756942 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:55.756946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:55.757017 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:55.757111 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:55.758074 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:55.759053 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:55.760938 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:55.761605 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:55.761771 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:55.766730 (+ 4959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:56925: BlockingRecv error: recv got EOF from 127.25.236.131:56925 (error 108)
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":361,"thread_start_us":266,"threads_started":1}
Aug 06 01:14:55 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:55 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:55.769753 8296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:55.759672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:43397)
0806 01:14:55.760250 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:55.760274 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:55.760386 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:55.760755 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:55.760759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:55.761249 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:55.761434 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:55.761445 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:55.763310 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:55.763314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:55.764272 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:55.764283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:55.765529 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:55.765537 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:55.765763 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:55.766385 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:55.766408 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:55.769470 (+ 3062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":275,"threads_started":1}
W20260806 01:14:55.770053 8298 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:55.759718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:43397 (local address 127.25.236.190:39649)
0806 01:14:55.760469 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:55.760474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:55.760491 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:14:55.760536 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:55.760540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:55.760617 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:55.760725 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:55.761639 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:55.763041 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:55.764494 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:55.765332 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:55.765447 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:55.769734 (+ 4287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:43397: BlockingRecv error: recv got EOF from 127.25.236.129:43397 (error 108)
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":631,"thread_start_us":508,"threads_started":1}
Aug 06 01:14:56 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:56 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:56.258311 8302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:56.244187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:36587)
0806 01:14:56.244789 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:56.244806 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:56.244922 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:56.245228 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:56.245232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:56.245609 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:56.245756 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:56.245764 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:56.247338 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:56.247343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:56.248411 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:56.248425 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:56.249256 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:56.249261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:56.249401 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:56.254915 (+ 5514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:56.254934 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:56.257913 (+ 2979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":360,"threads_started":1}
W20260806 01:14:56.258426 8298 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:56.244296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:36587 (local address 127.25.236.190:39649)
0806 01:14:56.244478 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:56.244483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:56.244880 (+ 397us) server_negotiation.cc:407] Connection header received
0806 01:14:56.245021 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:56.245025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:56.245088 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:56.245176 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:56.245912 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:56.247159 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:56.248564 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:56.248975 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:56.249100 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:56.258150 (+ 9050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:36587: BlockingRecv error: recv got EOF from 127.25.236.130:36587 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 01:14:56 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:56 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:56.779527 8303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:56.768182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:52233)
0806 01:14:56.768796 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:56.768822 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:56.768961 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:56.769516 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:56.769522 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:56.770297 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:56.770517 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:56.770529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:56.772296 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:56.772309 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:56.773414 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:56.773434 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:56.774277 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:56.774287 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:56.774460 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:56.775300 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:56.775318 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:56.779154 (+ 3836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":313,"threads_started":1}
Aug 06 01:14:56 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:14:56.780061 8304 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:56.768544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:52233 (local address 127.25.236.190:39649)
0806 01:14:56.769000 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:56.769004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:56.769020 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:14:56.769065 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:56.769068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:56.769216 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:56.769307 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:56.770680 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:56.772078 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:56.773588 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:56.774076 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:56.774181 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:56.779430 (+ 5249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:52233: BlockingRecv error: recv got EOF from 127.25.236.131:52233 (error 108)
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":234,"threads_started":1}
Aug 06 01:14:56 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:56.781849 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:56.770821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:54217)
0806 01:14:56.771531 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:56.771566 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:56.771673 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:56.772588 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:56.772592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:56.772963 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:56.773136 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:56.773145 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:56.774545 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:56.774554 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:56.775998 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:56.776014 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:56.776966 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:56.776973 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:56.777217 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:56.778061 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:56.778088 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:56.781482 (+ 3394us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":278,"threads_started":1}
W20260806 01:14:56.781951 8309 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:56.770891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:54217 (local address 127.25.236.190:39649)
0806 01:14:56.771676 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:56.771682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:56.771700 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:14:56.771942 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:56.771949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:56.772077 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:56.772235 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:56.773318 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:56.774206 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:56.776185 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:56.776715 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:56.776862 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:56.781694 (+ 4832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:54217: BlockingRecv error: recv got EOF from 127.25.236.129:54217 (error 108)
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":534,"threads_started":1}
Aug 06 01:14:57 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:57 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:57.268390 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:57.259526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:37797)
0806 01:14:57.260139 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:57.260156 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:57.260342 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:57.260705 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:57.260710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:57.261136 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:57.261302 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:57.261311 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:57.262516 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:57.262525 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:57.263446 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:57.263458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:57.264191 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:57.264209 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:57.264375 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:57.264994 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:57.265011 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:57.268005 (+ 2994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":301,"threads_started":1}
W20260806 01:14:57.268481 8309 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:57.259804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:37797 (local address 127.25.236.190:39649)
0806 01:14:57.260033 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:57.260040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:57.260258 (+ 218us) server_negotiation.cc:407] Connection header received
0806 01:14:57.260479 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:57.260484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:57.260560 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:57.260673 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:57.261545 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:57.262387 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:57.263597 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:57.264028 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:57.264163 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:57.268205 (+ 4042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:37797: BlockingRecv error: recv got EOF from 127.25.236.130:37797 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 06 01:14:57 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:57 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:57 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:57.790889 8313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:57.781275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:49455)
0806 01:14:57.781924 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:57.781941 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:57.782063 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:57.782518 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:57.782522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:57.783189 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:57.783466 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:57.783478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:57.784863 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:57.784868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:57.785866 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:57.785879 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:57.786696 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:57.786702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:57.786895 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:57.787529 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:57.787547 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:57.790565 (+ 3018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":262,"threads_started":1}
W20260806 01:14:57.791445 8314 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:57.781373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:49455 (local address 127.25.236.190:39649)
0806 01:14:57.781933 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:57.781937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:57.782020 (+ 83us) server_negotiation.cc:407] Connection header received
0806 01:14:57.782279 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:57.782289 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:57.782359 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:57.782486 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:57.783656 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:57.784702 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:57.786017 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:57.786548 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:57.786680 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:57.790844 (+ 4164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:49455: BlockingRecv error: recv got EOF from 127.25.236.131:49455 (error 108)
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":330,"threads_started":1}
Aug 06 01:14:57 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:57.792889 8315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:57.782825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:41249)
0806 01:14:57.783368 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:57.783396 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:57.783523 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:57.784024 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:57.784029 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:57.784814 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:57.785255 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:57.785266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:57.786353 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:57.786361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:57.787407 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:57.787420 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:57.788304 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:57.788311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:57.788517 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:57.789097 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:57.789115 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:57.792573 (+ 3458us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":298,"threads_started":1}
W20260806 01:14:57.793046 8316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:57.782855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:41249 (local address 127.25.236.190:39649)
0806 01:14:57.783690 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:57.783695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:57.783737 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:14:57.783794 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:57.783800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:57.783887 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:57.783992 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:57.785410 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:57.786186 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:57.787572 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:57.788008 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:57.788292 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:57.792775 (+ 4483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:41249: BlockingRecv error: recv got EOF from 127.25.236.129:41249 (error 108)
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":571,"threads_started":1}
Aug 06 01:14:58 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:58 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:58.280182 8318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:58.269819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:46215)
0806 01:14:58.270552 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:58.270570 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:58.270743 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:58.271082 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:58.271087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:58.271704 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:58.271921 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:58.271935 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:58.273333 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:58.273339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:58.274717 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:58.274730 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:58.275488 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:58.275493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:58.275616 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:58.276678 (+ 1062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:58.276697 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:58.279954 (+ 3257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":337,"threads_started":1}
W20260806 01:14:58.280508 8316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:58.269920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:46215 (local address 127.25.236.190:39649)
0806 01:14:58.270127 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:58.270136 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:58.270669 (+ 533us) server_negotiation.cc:407] Connection header received
0806 01:14:58.270872 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:58.270881 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:58.270943 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:58.271046 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:58.272135 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:58.273143 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:58.274884 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:58.275335 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:58.275440 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:58.280237 (+ 4797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:46215: BlockingRecv error: recv got EOF from 127.25.236.130:46215 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 01:14:58 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:58 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:58 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:58 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:58.803701 8320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:58.792448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:37277)
0806 01:14:58.793057 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:58.793082 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:58.793611 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:58.794522 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:58.794530 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:58.795222 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:58.795409 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:58.795418 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:58.796585 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:58.796598 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:58.797759 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:58.797775 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:58.798742 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:58.798750 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:58.798950 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:58.799752 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:58.799776 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:58.803363 (+ 3587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":327,"threads_started":1}
W20260806 01:14:58.803915 8322 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:58.792441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:37277 (local address 127.25.236.190:39649)
0806 01:14:58.793549 (+ 1108us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:58.793556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:58.793592 (+ 36us) server_negotiation.cc:407] Connection header received
0806 01:14:58.793928 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:58.793933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:58.794018 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:58.794223 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:58.795541 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:58.796413 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:58.797928 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:58.798565 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:58.798662 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:58.803638 (+ 4976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:37277: BlockingRecv error: recv got EOF from 127.25.236.131:37277 (error 108)
Metrics: {"server-negotiator.queue_time_us":994,"thread_start_us":853,"threads_started":1}
I20260806 01:14:58.804600 8324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:58.793930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:54525)
0806 01:14:58.795112 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:58.795139 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:58.795272 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:58.795802 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:58.795812 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:58.796174 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:58.796330 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:58.796338 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:58.797431 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:58.797437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:58.798793 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:58.798808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:58.799730 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:58.799741 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:58.799933 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:58.800602 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:58.800620 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:58.804350 (+ 3730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1043,"thread_start_us":851,"threads_started":1}
W20260806 01:14:58.804808 8323 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:58.793958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:54525 (local address 127.25.236.190:39649)
0806 01:14:58.795515 (+ 1557us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:58.795521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:58.795534 (+ 13us) server_negotiation.cc:407] Connection header received
0806 01:14:58.795572 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:58.795576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:58.795655 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:58.795762 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:58.796486 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:58.797263 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:58.798930 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:58.799446 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:58.799573 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:58.804554 (+ 4981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:54525: BlockingRecv error: recv got EOF from 127.25.236.129:54525 (error 108)
Metrics: {"server-negotiator.queue_time_us":1413,"thread_start_us":1191,"threads_started":1}
Aug 06 01:14:59 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:59 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:59.289436 8325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:59.281259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:36167)
0806 01:14:59.281794 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:59.281811 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:59.281968 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:59.282332 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:59.282337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:59.282779 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:59.282943 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:59.282951 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:59.283975 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:59.283982 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:59.284887 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:59.284898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:59.285569 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:59.285575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:59.285742 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:59.286408 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:59.286426 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:59.289199 (+ 2773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":242,"threads_started":1}
W20260806 01:14:59.289685 8323 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:59.281315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:36167 (local address 127.25.236.190:39649)
0806 01:14:59.281541 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:59.281549 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:59.281904 (+ 355us) server_negotiation.cc:407] Connection header received
0806 01:14:59.282090 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:59.282095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:59.282165 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:59.282298 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:59.283084 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:59.283834 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:59.285024 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:59.285419 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:59.285547 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:59.289403 (+ 3856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:36167: BlockingRecv error: recv got EOF from 127.25.236.130:36167 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 01:14:59 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:14:59 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:59.818941 8328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:59.804776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:42657)
0806 01:14:59.805800 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:59.805820 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:59.805983 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:59.806538 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:59.806543 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:59.807024 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:59.807203 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:59.807212 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:59.809147 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:59.809153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:59.810819 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:59.810835 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:59.811988 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:59.811998 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:59.812240 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:59.813866 (+ 1626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:59.813900 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:59.818531 (+ 4631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":863,"thread_start_us":596,"threads_started":1}
Aug 06 01:14:59 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:14:59.819891 8326 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:59.804898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:42657 (local address 127.25.236.190:39649)
0806 01:14:59.805568 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:59.805575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:59.805899 (+ 324us) server_negotiation.cc:407] Connection header received
0806 01:14:59.806148 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:59.806152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:59.806212 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:59.806522 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:59.807360 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:59.808799 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:59.811010 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:59.811624 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:59.811725 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:59.818999 (+ 7274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:42657: BlockingRecv error: recv got EOF from 127.25.236.131:42657 (error 108)
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":364,"threads_started":1}
Aug 06 01:14:59 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:14:59.822064 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:14:59.805852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:56539)
0806 01:14:59.807721 (+ 1869us) negotiation.cc:107] Waiting for socket to connect
0806 01:14:59.807748 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 01:14:59.807876 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:14:59.808739 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:14:59.808745 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:14:59.809280 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:14:59.809428 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:59.809436 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:59.811160 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:59.811165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:59.812709 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:14:59.812744 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:14:59.816363 (+ 3619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:14:59.816381 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:14:59.816634 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:59.817349 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:14:59.817376 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:14:59.821193 (+ 3817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1708,"thread_start_us":1456,"threads_started":1}
W20260806 01:14:59.824373 8330 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:14:59.806098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:56539 (local address 127.25.236.190:39649)
0806 01:14:59.808173 (+ 2075us) server_negotiation.cc:206] Beginning negotiation
0806 01:14:59.808182 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:14:59.808203 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:14:59.808257 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:14:59.808263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:14:59.808337 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:14:59.808462 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:14:59.809578 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:59.811009 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:59.812963 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:14:59.813636 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:14:59.813797 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:14:59.823747 (+ 9950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:56539: BlockingRecv error: recv got EOF from 127.25.236.129:56539 (error 108)
Metrics: {"server-negotiator.queue_time_us":1927,"thread_start_us":1746,"threads_started":1}
Aug 06 01:15:00 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:00 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:00.299391 8331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:00.290385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:46173)
0806 01:15:00.290911 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:00.290928 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:00.291048 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:00.291439 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:00.291443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:00.291842 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:00.292000 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:00.292010 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:00.293482 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:00.293487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:00.294608 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:00.294623 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:00.295510 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:00.295519 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:00.295652 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:00.296339 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:00.296357 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:00.299161 (+ 2804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":268,"threads_started":1}
W20260806 01:15:00.299679 8330 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:00.290447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:46173 (local address 127.25.236.190:39649)
0806 01:15:00.290702 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:00.290708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:00.291006 (+ 298us) server_negotiation.cc:407] Connection header received
0806 01:15:00.291172 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:00.291180 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:00.291258 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:00.291339 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:00.292249 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:00.293280 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:00.294765 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:00.295298 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:00.295405 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:00.299359 (+ 3954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:46173: BlockingRecv error: recv got EOF from 127.25.236.130:46173 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 01:15:00 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:00 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:00.829725 8333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:00.820059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:42573)
0806 01:15:00.820778 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:00.820795 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:00.820899 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:00.821282 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:00.821286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:00.821698 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:00.821862 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:00.821871 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:00.822911 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:00.822917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:00.824441 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:00.824453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:00.825383 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:00.825390 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:00.825554 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:00.826283 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:00.826305 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:00.829486 (+ 3181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":441,"threads_started":1}
W20260806 01:15:00.830030 8334 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:00.820373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:42573 (local address 127.25.236.190:39649)
0806 01:15:00.820850 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:00.820854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:00.820872 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:15:00.821003 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:00.821007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:00.821094 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:00.821193 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:00.821994 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:00.822760 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:00.824574 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:00.825206 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:00.825310 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:00.829733 (+ 4423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:42573: BlockingRecv error: recv got EOF from 127.25.236.131:42573 (error 108)
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":281,"threads_started":1}
Aug 06 01:15:00 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:00 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:00.832347 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:00.823016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:47659)
0806 01:15:00.823661 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:00.823686 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:00.823819 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:00.824147 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:00.824150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:00.824540 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:00.824794 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:00.824806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:00.826121 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:00.826129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:00.827321 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:00.827337 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:00.828371 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:00.828378 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:00.828558 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:00.829182 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:00.829209 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:00.832126 (+ 2917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":353,"threads_started":1}
W20260806 01:15:00.832576 8339 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:00.823092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:47659 (local address 127.25.236.190:39649)
0806 01:15:00.823879 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:00.823885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:00.823903 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:15:00.823945 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:00.823950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:00.824009 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:00.824114 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:00.824936 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:00.825968 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:00.827484 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:00.828199 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:00.828319 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:00.832304 (+ 3985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:47659: BlockingRecv error: recv got EOF from 127.25.236.129:47659 (error 108)
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":454,"threads_started":1}
Aug 06 01:15:01 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:01 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:01.316720 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:01.300872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:35449)
0806 01:15:01.301711 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:01.301734 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:01.301888 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:01.302281 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:01.302286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:01.302944 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:01.303280 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:01.303307 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:01.304709 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:01.304716 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:01.306006 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:01.306028 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:01.307058 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:01.307066 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:01.307299 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:01.311586 (+ 4287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:01.311610 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:01.316317 (+ 4707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":528,"threads_started":1}
W20260806 01:15:01.316991 8339 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:01.300956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:35449 (local address 127.25.236.190:39649)
0806 01:15:01.301228 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:01.301239 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:01.301813 (+ 574us) server_negotiation.cc:407] Connection header received
0806 01:15:01.302032 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:01.302038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:01.302113 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:01.302257 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:01.303555 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:01.304494 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:01.306315 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:01.306825 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:01.307003 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:01.316519 (+ 9516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:35449: BlockingRecv error: recv got EOF from 127.25.236.130:35449 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 01:15:01 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:01 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:01 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:01 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:01.843504 8342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:01.831280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:40741)
0806 01:15:01.832572 (+ 1292us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:01.832597 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:01.832788 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:01.833391 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:01.833395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:01.833986 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:01.834396 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:01.834418 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:01.835964 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:01.835971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:01.837330 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:01.837343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:01.838341 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:01.838346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:01.838467 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:01.839114 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:01.839209 (+ 95us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:01.843134 (+ 3925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1141,"thread_start_us":971,"threads_started":1}
W20260806 01:15:01.843537 8343 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:01.831609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:40741 (local address 127.25.236.190:39649)
0806 01:15:01.832668 (+ 1059us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:01.832675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:01.832701 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:15:01.832938 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:01.832944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:01.833058 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:01.833368 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:01.834576 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:01.835821 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:01.837477 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:01.838114 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:01.838271 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:01.843338 (+ 5067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:40741: BlockingRecv error: recv got EOF from 127.25.236.131:40741 (error 108)
Metrics: {"server-negotiator.queue_time_us":906,"thread_start_us":754,"threads_started":1}
I20260806 01:15:01.844027 8344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:01.833363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:55969)
0806 01:15:01.833929 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:01.833963 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:01.834110 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:01.834788 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:01.834792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:01.835376 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:01.835550 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:01.835560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:01.837173 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:01.837184 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:01.838561 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:01.838572 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:01.839614 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:01.839623 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:01.839752 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:01.840337 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:01.840354 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:01.843721 (+ 3367us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":360,"threads_started":1}
W20260806 01:15:01.844283 8347 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:01.833669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:55969 (local address 127.25.236.190:39649)
0806 01:15:01.834495 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:01.834503 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:01.834521 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:15:01.834566 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:01.834570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:01.834625 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:01.834710 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:01.835719 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:01.836940 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:01.838703 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:01.839426 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:01.839576 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:01.843952 (+ 4376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:55969: BlockingRecv error: recv got EOF from 127.25.236.129:55969 (error 108)
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":387,"threads_started":1}
Aug 06 01:15:02 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:02 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:15:02.332286 8347 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:02.318265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:43573 (local address 127.25.236.190:39649)
0806 01:15:02.319188 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:02.319197 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:02.319362 (+ 165us) server_negotiation.cc:407] Connection header received
0806 01:15:02.319612 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:02.319619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:02.319717 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:02.319828 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:02.321290 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:02.322794 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:02.325186 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:02.325885 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:02.326033 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:02.331963 (+ 5930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:43573: BlockingRecv error: recv got EOF from 127.25.236.130:43573 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:15:02.332927 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:02.318123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:43573)
0806 01:15:02.319187 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:02.319211 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:02.319403 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:02.319888 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:02.319896 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:02.320515 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:02.320836 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:02.320850 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:02.323561 (+ 2711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:02.323566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:02.325006 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:02.325023 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:02.326099 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:02.326104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:02.326594 (+ 490us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:02.327382 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:02.327412 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:02.332545 (+ 5133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":923,"thread_start_us":783,"threads_started":1}
Aug 06 01:15:02 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:02 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:02 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:02 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:02.858919 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:02.845168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:40663)
0806 01:15:02.848831 (+ 3663us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:02.848867 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:02.849057 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:02.850154 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:02.850161 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:02.850621 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:02.850802 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:02.850811 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:02.851904 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:02.851909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:02.852916 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:02.852927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:02.853913 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:02.853920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:02.854097 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:02.854838 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:02.854866 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:02.858551 (+ 3685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":3470,"thread_start_us":3326,"threads_started":1}
I20260806 01:15:02.859097 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:02.845064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:36313)
0806 01:15:02.849264 (+ 4200us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:02.849292 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:02.849436 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:02.849933 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:02.849938 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:02.850421 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:02.850569 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:02.850577 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:02.851670 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:02.851678 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:02.852954 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:02.852970 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:02.854474 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:02.854479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:02.854678 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:02.855219 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:02.855246 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:02.858757 (+ 3511us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":3967,"thread_start_us":3858,"threads_started":1}
W20260806 01:15:02.859097 8354 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:02.845127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:40663 (local address 127.25.236.190:39649)
0806 01:15:02.849874 (+ 4747us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:02.849878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:02.849893 (+ 15us) server_negotiation.cc:407] Connection header received
0806 01:15:02.849932 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:02.849935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:02.849998 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:02.850117 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:02.850945 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:02.851775 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:02.853048 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:02.853741 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:02.853861 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:02.858772 (+ 4911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:40663: BlockingRecv error: recv got EOF from 127.25.236.129:40663 (error 108)
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":4655,"thread_start_us":4448,"threads_started":1}
W20260806 01:15:02.859623 8353 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:02.845110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:36313 (local address 127.25.236.190:39649)
0806 01:15:02.849589 (+ 4479us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:02.849595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:02.849610 (+ 15us) server_negotiation.cc:407] Connection header received
0806 01:15:02.849659 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:02.849663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:02.849767 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:02.849893 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:02.850739 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:02.851521 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:02.853295 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:02.854233 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:02.854376 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:02.859027 (+ 4651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:36313: BlockingRecv error: recv got EOF from 127.25.236.131:36313 (error 108)
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":4337,"thread_start_us":4200,"threads_started":1}
Aug 06 01:15:03 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:03 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:03.347463 8357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:03.333932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:57197)
0806 01:15:03.338657 (+ 4725us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:03.338676 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:03.338805 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:03.339207 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:03.339211 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:03.339627 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:03.339818 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:03.339826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:03.340849 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:03.340856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:03.341900 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:03.341913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:03.343005 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:03.343010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:03.343257 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:03.343886 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:03.343904 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:03.347105 (+ 3201us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4619,"thread_start_us":4479,"threads_started":1}
W20260806 01:15:03.347564 8353 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:03.334019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:57197 (local address 127.25.236.190:39649)
0806 01:15:03.334222 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:03.334226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:03.338772 (+ 4546us) server_negotiation.cc:407] Connection header received
0806 01:15:03.338910 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:03.338915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:03.338978 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:03.339061 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:03.339958 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:03.340704 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:03.342105 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:03.342844 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:03.342956 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:03.347316 (+ 4360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:57197: BlockingRecv error: recv got EOF from 127.25.236.130:57197 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 01:15:03 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:03 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:03 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:03 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:03.870153 8362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:03.860284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:35561)
0806 01:15:03.860825 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:03.860845 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:03.860993 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:03.862009 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:03.862014 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:03.862571 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:03.862903 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:03.862916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:03.864343 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:03.864349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:03.865221 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:03.865232 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:03.866074 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:03.866081 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:03.866221 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:03.866928 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:03.866946 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:03.869914 (+ 2968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":440,"threads_started":1}
W20260806 01:15:03.870457 8363 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:03.860636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:35561 (local address 127.25.236.190:39649)
0806 01:15:03.861737 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:03.861742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:03.861756 (+ 14us) server_negotiation.cc:407] Connection header received
0806 01:15:03.861793 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:03.861797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:03.861846 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:03.861972 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:03.863070 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:03.864222 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:03.865382 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:03.865885 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:03.866028 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:03.870145 (+ 4117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:35561: BlockingRecv error: recv got EOF from 127.25.236.131:35561 (error 108)
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":1007,"thread_start_us":884,"threads_started":1}
I20260806 01:15:03.870678 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:03.860090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:41199)
0806 01:15:03.861153 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:03.861175 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:03.861317 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:03.861660 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:03.861665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:03.862197 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:03.862431 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:03.862444 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:03.864111 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:03.864141 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:03.865168 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:03.865180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:03.866003 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:03.866024 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:03.866250 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:03.867019 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:03.867037 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:03.870351 (+ 3314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":951,"thread_start_us":847,"threads_started":1}
W20260806 01:15:03.870945 8361 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:03.860126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:41199 (local address 127.25.236.190:39649)
0806 01:15:03.860763 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:03.860773 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:03.861261 (+ 488us) server_negotiation.cc:407] Connection header received
0806 01:15:03.861430 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:03.861437 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:03.861510 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:03.861631 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:03.862593 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:03.863939 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:03.865312 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:03.865853 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:03.865947 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:03.870658 (+ 4711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:41199: BlockingRecv error: recv got EOF from 127.25.236.129:41199 (error 108)
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":466,"thread_start_us":280,"threads_started":1}
Aug 06 01:15:04 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:04 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:04.357900 8365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:04.348661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:39915)
0806 01:15:04.349275 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:04.349306 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:04.349434 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:04.349778 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:04.349782 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:04.350180 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:04.350380 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:04.350388 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:04.351564 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:04.351568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:04.352601 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:04.352615 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:04.353547 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:04.353553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:04.353747 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:04.354494 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:04.354524 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:04.357556 (+ 3032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":284,"threads_started":1}
W20260806 01:15:04.358047 8361 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:04.348755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:39915 (local address 127.25.236.190:39649)
0806 01:15:04.348999 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:04.349007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:04.349411 (+ 404us) server_negotiation.cc:407] Connection header received
0806 01:15:04.349541 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:04.349545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:04.349617 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:04.349706 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:04.350535 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:04.351401 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:04.352811 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:04.353355 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:04.353524 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:04.357788 (+ 4264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:39915: BlockingRecv error: recv got EOF from 127.25.236.130:39915 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 01:15:04 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:04 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:04 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:04 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:04.883180 8367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:04.871364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:54555)
0806 01:15:04.871960 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:04.871985 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:04.872123 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:04.872963 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:04.872968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:04.873565 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:04.873729 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:04.873738 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:04.875523 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:04.875531 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:04.877100 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:04.877115 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:04.878223 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:04.878285 (+ 62us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:04.878454 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:04.879156 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:04.879183 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:04.882843 (+ 3660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":343,"threads_started":1}
W20260806 01:15:04.883699 8370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:04.871447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:54555 (local address 127.25.236.190:39649)
0806 01:15:04.872277 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:04.872284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:04.872304 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:15:04.872678 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:04.872684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:04.872781 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:04.872909 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:04.873867 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:04.875238 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:04.877256 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:04.877914 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:04.878213 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:04.883191 (+ 4978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:54555: BlockingRecv error: recv got EOF from 127.25.236.131:54555 (error 108)
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":547,"threads_started":1}
W20260806 01:15:04.884452 8376 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:04.872102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:46599 (local address 127.25.236.190:39649)
0806 01:15:04.872724 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:04.872728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:04.873039 (+ 311us) server_negotiation.cc:407] Connection header received
0806 01:15:04.873300 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:04.873308 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:04.873399 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:04.873518 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:04.874901 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:04.876555 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:04.878652 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:04.879289 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:04.879439 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:04.884087 (+ 4648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:46599: BlockingRecv error: recv got EOF from 127.25.236.129:46599 (error 108)
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":481,"thread_start_us":299,"threads_started":1}
I20260806 01:15:04.884471 8373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:04.871688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:46599)
0806 01:15:04.872794 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:04.872823 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:04.872940 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:04.873550 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:04.873557 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:04.874075 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:04.874615 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:04.874638 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:04.876742 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:04.876747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:04.878230 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:04.878364 (+ 134us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:04.879523 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:04.879528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:04.879653 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:04.880392 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:04.880410 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:04.883853 (+ 3443us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":578,"threads_started":1}
Aug 06 01:15:05 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:05 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:05.367388 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:05.359007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:44019)
0806 01:15:05.359618 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:05.359634 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:05.359760 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:05.360055 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:05.360058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:05.360458 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:05.360620 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:05.360628 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:05.361708 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:05.361712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:05.362710 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:05.362722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:05.363489 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:05.363493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:05.363604 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:05.364200 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:05.364217 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:05.367160 (+ 2943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":329,"threads_started":1}
W20260806 01:15:05.367663 8376 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:05.359196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:44019 (local address 127.25.236.190:39649)
0806 01:15:05.359392 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:05.359396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:05.359734 (+ 338us) server_negotiation.cc:407] Connection header received
0806 01:15:05.359863 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:05.359867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:05.359929 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:05.360032 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:05.360795 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:05.361562 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:05.362846 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:05.363326 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:05.363456 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:05.367374 (+ 3918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:44019: BlockingRecv error: recv got EOF from 127.25.236.130:44019 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 06 01:15:05 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:05 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:05 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:05 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:05.899288 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:05.885393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:38745)
0806 01:15:05.885997 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:05.886022 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:05.886132 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:05.886803 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:05.886813 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:05.887536 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:05.887753 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:05.887763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:05.889366 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:05.889376 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:05.891345 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:05.891361 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:05.893018 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:05.893029 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:05.893285 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:05.894207 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:05.894385 (+ 178us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:05.898915 (+ 4530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":380,"threads_started":1}
W20260806 01:15:05.899492 8382 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:05.885438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:38745 (local address 127.25.236.190:39649)
0806 01:15:05.886386 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:05.886391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:05.886405 (+ 14us) server_negotiation.cc:407] Connection header received
0806 01:15:05.886441 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:05.886444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:05.886490 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:05.886577 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:05.887985 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:05.889130 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:05.891514 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:05.892736 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:05.893011 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:05.899140 (+ 6129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:38745: BlockingRecv error: recv got EOF from 127.25.236.129:38745 (error 108)
Metrics: {"mutex_wait_us":236,"server-negotiator.queue_time_us":872,"thread_start_us":654,"threads_started":1}
I20260806 01:15:05.899821 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:05.884415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:52727)
0806 01:15:05.885378 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:05.885394 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:05.885549 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:05.886116 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:05.886121 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:05.887113 (+ 992us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:05.887393 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:05.887408 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:05.889062 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:05.889066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:05.891132 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:05.891149 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:05.893099 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:05.893107 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:05.893328 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:05.894370 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:05.894411 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:05.899523 (+ 5112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":833,"thread_start_us":1117,"threads_started":1}
W20260806 01:15:05.900249 8379 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:05.884527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:52727 (local address 127.25.236.190:39649)
0806 01:15:05.885284 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:05.885291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:05.885477 (+ 186us) server_negotiation.cc:407] Connection header received
0806 01:15:05.885679 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:05.885685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:05.885794 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:05.885906 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:05.887587 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:05.888913 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:05.891514 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:05.892736 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:05.892951 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:05.899965 (+ 7014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:52727: BlockingRecv error: recv got EOF from 127.25.236.131:52727 (error 108)
Metrics: {"mutex_wait_us":182,"server-negotiator.queue_time_us":626,"thread_start_us":498,"threads_started":1}
Aug 06 01:15:06 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:06 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:06.381665 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:06.368539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:36473)
0806 01:15:06.369122 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:06.369165 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:06.369327 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:06.369673 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:06.369678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:06.370079 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:06.370287 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:06.370296 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:06.371354 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:06.371360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:06.372262 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:06.372274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:06.372949 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:06.372955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:06.373128 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:06.377918 (+ 4790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:06.377938 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:06.381408 (+ 3470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":291,"threads_started":1}
W20260806 01:15:06.381950 8379 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:06.368472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:36473 (local address 127.25.236.190:39649)
0806 01:15:06.368647 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:06.368651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:06.369256 (+ 605us) server_negotiation.cc:407] Connection header received
0806 01:15:06.369456 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:06.369461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:06.369533 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:06.369635 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:06.370433 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:06.371211 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:06.372405 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:06.372790 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:06.372926 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:06.381651 (+ 8725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:36473: BlockingRecv error: recv got EOF from 127.25.236.130:36473 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 01:15:06 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:06 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:06 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.131@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:06 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.129@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:06.913601 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:06.900927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.131:45621)
0806 01:15:06.902393 (+ 1466us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:06.902413 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:06.902528 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:06.902806 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:06.902810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:06.903351 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:06.903584 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:06.903596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:06.905432 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:06.905437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:06.906975 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:06.906991 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:06.908215 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:06.908221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:06.908391 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:06.909458 (+ 1067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:06.909483 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:06.913350 (+ 3867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1329,"thread_start_us":1205,"threads_started":1}
W20260806 01:15:06.914104 8389 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:06.901169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:45621 (local address 127.25.236.190:39649)
0806 01:15:06.902043 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:06.902051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:06.902473 (+ 422us) server_negotiation.cc:407] Connection header received
0806 01:15:06.902605 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:06.902609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:06.902673 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:06.902763 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:06.903758 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:06.905265 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:06.907160 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:06.907986 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:06.908090 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:06.913811 (+ 5721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.131:45621: BlockingRecv error: recv got EOF from 127.25.236.131:45621 (error 108)
Metrics: {"mutex_wait_us":738,"server-negotiator.queue_time_us":697,"thread_start_us":428,"threads_started":1}
I20260806 01:15:06.914568 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:06.900674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.129:42387)
0806 01:15:06.902065 (+ 1391us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:06.902096 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:06.902279 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:06.902796 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:06.902801 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:06.903349 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:06.903584 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:06.903596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:06.905424 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:06.905430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:06.906991 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:06.907005 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:06.908196 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:06.908202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:06.908372 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:06.909458 (+ 1086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:06.909483 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:06.914195 (+ 4712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1257,"thread_start_us":1151,"threads_started":1}
W20260806 01:15:06.914901 8385 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:06.900731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:42387 (local address 127.25.236.190:39649)
0806 01:15:06.901342 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:06.901353 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:06.902193 (+ 840us) server_negotiation.cc:407] Connection header received
0806 01:15:06.902486 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:06.902494 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:06.902602 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:06.902723 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:06.903758 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:06.905265 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:06.907160 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:06.907960 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:06.908094 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:06.914648 (+ 6554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.129:42387: BlockingRecv error: recv got EOF from 127.25.236.129:42387 (error 108)
Metrics: {"mutex_wait_us":211,"server-negotiator.queue_time_us":394,"thread_start_us":236,"threads_started":1}
Aug 06 01:15:07 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:15:07 dist-test-slave-7nm7 krb5kdc[6711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.236.130@KRBTEST.COM for oryx/127.25.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:15:07.394099 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:07.383026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39649 (local address 127.25.236.130:49985)
0806 01:15:07.383581 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:07.383597 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:07.383765 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:07.384133 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:07.384136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:07.384689 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:07.384938 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:07.384952 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:07.386603 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:07.386607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:07.387936 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:07.387966 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:07.389043 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:07.389048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:07.389288 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:07.390201 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:07.390222 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:07.393832 (+ 3610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.236.190:39649: Server oryx/127.25.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":320,"threads_started":1}
W20260806 01:15:07.394677 8385 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:07.383209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:49985 (local address 127.25.236.190:39649)
0806 01:15:07.383393 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:07.383398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:07.383675 (+ 277us) server_negotiation.cc:407] Connection header received
0806 01:15:07.383883 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:07.383887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:07.383948 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:07.384072 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:07.385152 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:07.386391 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:07.388255 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:07.388775 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:07.388989 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:07.394131 (+ 5142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.236.130:49985: BlockingRecv error: recv got EOF from 127.25.236.130:49985 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:15:07.696979 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 7630
I20260806 01:15:07.711580 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:07.698226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42538 (local address 127.25.236.129:40259)
0806 01:15:07.699191 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:07.699199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:07.699230 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:15:07.699435 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:07.699443 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:07.699698 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:07.699859 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:07.701537 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:07.702829 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:07.704160 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:07.704758 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:07.704896 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:07.708065 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:07.708076 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:07.708092 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:15:07.708225 (+ 133us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:07.710999 (+ 2774us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.25.236.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 01:15:07.711339 (+ 340us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42538: Request ticket server kudu/127.25.236.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":587,"threads_started":1}
W20260806 01:15:07.711673 8394 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42538: Request ticket server kudu/127.25.236.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 01:15:07.712350 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 7630: Not authorized: Client connection negotiation failed: client connection to 127.25.236.129:40259: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.25.236.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 01:15:07.712680 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 7630
I20260806 01:15:07.746846 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 7877
I20260806 01:15:07.763176 8396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:07.748595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50356 (local address 127.25.236.130:43607)
0806 01:15:07.749577 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:07.749587 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:07.749629 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:15:07.749710 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:07.749718 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:07.750343 (+ 625us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:07.750560 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:07.752044 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:07.753515 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:07.755728 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:07.756526 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:07.756666 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:07.759724 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:07.759733 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:07.759748 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:15:07.759789 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:07.762640 (+ 2851us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.25.236.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 01:15:07.762926 (+ 286us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50356: Request ticket server kudu/127.25.236.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":455,"threads_started":1}
W20260806 01:15:07.763301 8396 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50356: Request ticket server kudu/127.25.236.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 01:15:07.763448 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 7877: Not authorized: Client connection negotiation failed: client connection to 127.25.236.130:43607: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.25.236.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 01:15:07.763664 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 7877
I20260806 01:15:07.794178 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 8081
I20260806 01:15:07.806707 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:07.795262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54384 (local address 127.25.236.131:34707)
0806 01:15:07.795861 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:07.795869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:07.795897 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:15:07.795958 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:07.795965 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:07.796187 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:07.796354 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:07.797712 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:07.798796 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:07.799977 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:07.800532 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:07.800637 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:07.803304 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:07.803313 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:07.803327 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:15:07.803356 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:07.805895 (+ 2539us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.25.236.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 01:15:07.806148 (+ 253us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54384: Request ticket server kudu/127.25.236.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":312,"threads_started":1}
W20260806 01:15:07.806813 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 8081: Not authorized: Client connection negotiation failed: client connection to 127.25.236.131:34707: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.25.236.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 01:15:07.806851 8397 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54384: Request ticket server kudu/127.25.236.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 01:15:07.806991 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 8081
I20260806 01:15:07.841096 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 7523
I20260806 01:15:07.842850 7620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:15:07.983209 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 7523
2026-08-06T01:15:08Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22177 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T01:15:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:15:08Z Disabled control of system clock
I20260806 01:15:08.100713 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:32885
--webserver_interface=127.25.236.190
--webserver_port=0
--builtin_ntp_servers=127.25.236.148:38449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:32885
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 01:15:08.490494 8407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:15:08.490839 8407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:15:08.490954 8407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:15:08.499617 8407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:15:08.499747 8407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:15:08.499787 8407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:15:08.499840 8407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:15:08.499888 8407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:15:08.510195 8407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:38449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:32885
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:32885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:15:08.511555 8407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:15:08.513412 8407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:15:08.525979 8413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:08.525941 8414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:08.526165 8417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:15:08.526706 8407 server_base.cc:1034] running on GCE node
I20260806 01:15:08.527540 8407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:15:08.528990 8407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:15:08.530292 8407 hybrid_clock.cc:648] HybridClock initialized: now 1785978908530160 us; error 61 us; skew 500 ppm
I20260806 01:15:08.530730 8407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:15:08.533147 8407 webserver.cc:460] Webserver started at http://127.25.236.190:46021/ using document root <none> and password file <none>
I20260806 01:15:08.533888 8407 fs_manager.cc:359] Metadata directory not provided
I20260806 01:15:08.534004 8407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:15:08.534417 8407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:15:08.537503 8407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "fbe5bb443f8942cba09c1b8b22489966"
format_stamp: "Formatted at 2026-08-06 01:15:08 on dist-test-slave-7nm7"
server_key: "14a9f68bd1a2ca4cd44cfb179ea76ae2"
server_key_iv: "906caa840c8ded5811266c5d88e1ca6c"
server_key_version: "encryptionkey@0"
I20260806 01:15:08.538575 8407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "fbe5bb443f8942cba09c1b8b22489966"
format_stamp: "Formatted at 2026-08-06 01:15:08 on dist-test-slave-7nm7"
server_key: "14a9f68bd1a2ca4cd44cfb179ea76ae2"
server_key_iv: "906caa840c8ded5811266c5d88e1ca6c"
server_key_version: "encryptionkey@0"
I20260806 01:15:08.544214 8407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:15:08.548784 8423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:08.551568 8407 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 01:15:08.551895 8407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "fbe5bb443f8942cba09c1b8b22489966"
format_stamp: "Formatted at 2026-08-06 01:15:08 on dist-test-slave-7nm7"
server_key: "14a9f68bd1a2ca4cd44cfb179ea76ae2"
server_key_iv: "906caa840c8ded5811266c5d88e1ca6c"
server_key_version: "encryptionkey@0"
I20260806 01:15:08.552243 8407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:15:08.571360 8407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:15:08.572472 8407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:15:08.572809 8407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:15:08.594106 8407 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:32885
I20260806 01:15:08.594115 8503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:32885 every 8 connection(s)
I20260806 01:15:08.595571 8407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:15:08.599598 8505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:15:08.604035 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 8407
I20260806 01:15:08.604251 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:15:08.605155 26546 external_mini_cluster.cc:1428] Setting key 3e83dca1fb88e066fe66d13db48d40c8
I20260806 01:15:08.611359 8505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966: Bootstrap starting.
I20260806 01:15:08.615283 8505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966: Neither blocks nor log segments found. Creating new log.
I20260806 01:15:08.616580 8505 log.cc:824] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966: Log is configured to *not* fsync() on all Append() calls
I20260806 01:15:08.625264 8505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966: No bootstrap required, opened a new log
I20260806 01:15:08.626396 8508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:08.607423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48174 (local address 127.25.236.190:32885)
0806 01:15:08.608484 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:08.608494 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:08.608539 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:15:08.610474 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:08.610496 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:08.610931 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:08.611253 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:08.611993 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:08.613567 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:08.615547 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:08.616326 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:08.616475 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:08.620634 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:08.620673 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:08.620752 (+ 79us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:08.622060 (+ 1308us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:08.622338 (+ 278us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:08.622417 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:08.622662 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:08.623135 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 01:15:08.623801 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":397,"threads_started":1}
I20260806 01:15:08.658100 8505 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe5bb443f8942cba09c1b8b22489966" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 32885 } }
I20260806 01:15:08.660211 8505 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:15:08.660580 8505 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe5bb443f8942cba09c1b8b22489966, State: Initialized, Role: FOLLOWER
I20260806 01:15:08.663581 8505 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe5bb443f8942cba09c1b8b22489966" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 32885 } }
I20260806 01:15:08.664866 8505 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:15:08.665580 8505 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:15:08.666626 8505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:15:08.674140 8505 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe5bb443f8942cba09c1b8b22489966" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 32885 } }
I20260806 01:15:08.675964 8505 leader_election.cc:302] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbe5bb443f8942cba09c1b8b22489966; no voters:
I20260806 01:15:08.678635 8505 leader_election.cc:288] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:15:08.678812 8511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:15:08.683022 8511 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [term 1 LEADER]: Becoming Leader. State: Replica: fbe5bb443f8942cba09c1b8b22489966, State: Running, Role: LEADER
I20260806 01:15:08.685124 8511 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe5bb443f8942cba09c1b8b22489966" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 32885 } }
I20260806 01:15:08.688467 8505 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:15:08.697732 8515 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbe5bb443f8942cba09c1b8b22489966" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe5bb443f8942cba09c1b8b22489966" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 32885 } } }
I20260806 01:15:08.698706 8513 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbe5bb443f8942cba09c1b8b22489966. Latest consensus state: current_term: 1 leader_uuid: "fbe5bb443f8942cba09c1b8b22489966" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe5bb443f8942cba09c1b8b22489966" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 32885 } } }
I20260806 01:15:08.700793 8515 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [sys.catalog]: This master's current role is: LEADER
I20260806 01:15:08.701180 8513 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966 [sys.catalog]: This master's current role is: LEADER
I20260806 01:15:08.708586 8521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:15:08.733109 8521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:15:08.768666 8521 catalog_manager.cc:1347] Generated new cluster ID: 5c9dd43daa9d4f348dcb3b5c7679bd15
I20260806 01:15:08.769193 8521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:15:08.805351 8521 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:15:08.809617 8521 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:15:08.853348 8521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbe5bb443f8942cba09c1b8b22489966: Generated new TSK 0
I20260806 01:15:08.856387 8521 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:15:08.884817 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:32885
--builtin_ntp_servers=127.25.236.148:38449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 01:15:10.034019 8544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:15:10.034857 8544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:15:10.035122 8544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:15:10.053408 8544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:15:10.053684 8544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:15:10.054015 8544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:15:10.071808 8544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:38449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:32885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:15:10.074410 8544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:15:10.077924 8544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:15:10.103797 8550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:10.103996 8553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:10.106097 8556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:15:10.106948 8544 server_base.cc:1034] running on GCE node
I20260806 01:15:10.108843 8544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:15:10.111730 8544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:15:10.113515 8544 hybrid_clock.cc:648] HybridClock initialized: now 1785978910113418 us; error 73 us; skew 500 ppm
I20260806 01:15:10.114208 8544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:15:10.120653 8544 webserver.cc:460] Webserver started at http://127.25.236.129:40861/ using document root <none> and password file <none>
I20260806 01:15:10.122824 8544 fs_manager.cc:359] Metadata directory not provided
I20260806 01:15:10.123188 8544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:15:10.123994 8544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:15:10.132197 8544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "9aab01467f9e431689f4fc6ecda2769c"
format_stamp: "Formatted at 2026-08-06 01:15:10 on dist-test-slave-7nm7"
server_key: "b44be338a5062f8607b4bd38a7b11fea"
server_key_iv: "8e2ec4ebe139cf643eb256be60fe6d3d"
server_key_version: "encryptionkey@0"
I20260806 01:15:10.135701 8544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "9aab01467f9e431689f4fc6ecda2769c"
format_stamp: "Formatted at 2026-08-06 01:15:10 on dist-test-slave-7nm7"
server_key: "b44be338a5062f8607b4bd38a7b11fea"
server_key_iv: "8e2ec4ebe139cf643eb256be60fe6d3d"
server_key_version: "encryptionkey@0"
I20260806 01:15:10.149890 8544 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.013s sys 0.000s
I20260806 01:15:10.158840 8566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:10.162896 8544 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.006s sys 0.000s
I20260806 01:15:10.163236 8544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "9aab01467f9e431689f4fc6ecda2769c"
format_stamp: "Formatted at 2026-08-06 01:15:10 on dist-test-slave-7nm7"
server_key: "b44be338a5062f8607b4bd38a7b11fea"
server_key_iv: "8e2ec4ebe139cf643eb256be60fe6d3d"
server_key_version: "encryptionkey@0"
I20260806 01:15:10.163843 8544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:15:10.209792 8544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:15:10.211380 8544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:15:10.211925 8544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:15:10.213933 8544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:15:10.217092 8544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:15:10.217625 8544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:10.217973 8544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:15:10.218536 8544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:10.269913 8544 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:44357
I20260806 01:15:10.269960 8739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:44357 every 8 connection(s)
I20260806 01:15:10.271561 8544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:15:10.277772 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 8544
I20260806 01:15:10.277979 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:15:10.278856 26546 external_mini_cluster.cc:1428] Setting key 9e61c9128f2c05ac2d9e97128d9b35c0
I20260806 01:15:10.283552 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:32885
--builtin_ntp_servers=127.25.236.148:38449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 01:15:10.288558 8742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:10.274650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:37809 (local address 127.25.236.190:32885)
0806 01:15:10.275162 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:10.275166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:10.276465 (+ 1299us) server_negotiation.cc:407] Connection header received
0806 01:15:10.279052 (+ 2587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:10.279064 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:10.279157 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:10.279259 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:10.280597 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:10.281779 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:10.285050 (+ 3271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:10.285615 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:10.285726 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:10.287568 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:10.287573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:10.287576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:10.287604 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:10.287658 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:10.287671 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:10.287754 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:10.288070 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 01:15:10.288248 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":285,"threads_started":1}
I20260806 01:15:10.289939 8744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:10.275371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:32885 (local address 127.25.236.129:37809)
0806 01:15:10.276288 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:10.276344 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:10.278644 (+ 2300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:10.279555 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:10.279569 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:10.279675 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:10.280411 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:10.280431 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:10.281963 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:10.281969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:10.284857 (+ 2888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:10.284879 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:10.285884 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:10.285894 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:10.286266 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:10.287137 (+ 871us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:15:10.287143 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:10.287200 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:15:10.287313 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:15:10.287782 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:15:10.287796 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:15:10.287807 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 01:15:10.287995 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 01:15:10.288457 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":314,"threads_started":1}
I20260806 01:15:10.291980 8740 heartbeater.cc:347] Connected to a master server at 127.25.236.190:32885
I20260806 01:15:10.292325 8740 heartbeater.cc:464] Registering TS with master...
I20260806 01:15:10.293305 8740 heartbeater.cc:511] Master 127.25.236.190:32885 requested a full tablet report, sending...
I20260806 01:15:10.295662 8448 ts_manager.cc:194] Registered new tserver with Master: 9aab01467f9e431689f4fc6ecda2769c (127.25.236.129:44357)
I20260806 01:15:10.297804 8448 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.236.129:37809
W20260806 01:15:10.669569 8746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:15:10.669848 8746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:15:10.669940 8746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:15:10.677987 8746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:15:10.678185 8746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:15:10.678391 8746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:15:10.688402 8746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:38449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:32885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:15:10.689785 8746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:15:10.691965 8746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:15:10.705088 8752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:10.705164 8755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:10.705189 8753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:15:10.706045 8746 server_base.cc:1034] running on GCE node
I20260806 01:15:10.706749 8746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:15:10.707685 8746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:15:10.708909 8746 hybrid_clock.cc:648] HybridClock initialized: now 1785978910708858 us; error 40 us; skew 500 ppm
I20260806 01:15:10.709286 8746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:15:10.711633 8746 webserver.cc:460] Webserver started at http://127.25.236.130:46789/ using document root <none> and password file <none>
I20260806 01:15:10.712342 8746 fs_manager.cc:359] Metadata directory not provided
I20260806 01:15:10.712466 8746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:15:10.712829 8746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:15:10.716099 8746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "31c653670dd34377a4ed273f88a14720"
format_stamp: "Formatted at 2026-08-06 01:15:10 on dist-test-slave-7nm7"
server_key: "58e9fc8cf976c9301808418e0ce22521"
server_key_iv: "23ca99f154bbf936eb4567c959f031a1"
server_key_version: "encryptionkey@0"
I20260806 01:15:10.716990 8746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "31c653670dd34377a4ed273f88a14720"
format_stamp: "Formatted at 2026-08-06 01:15:10 on dist-test-slave-7nm7"
server_key: "58e9fc8cf976c9301808418e0ce22521"
server_key_iv: "23ca99f154bbf936eb4567c959f031a1"
server_key_version: "encryptionkey@0"
I20260806 01:15:10.723027 8746 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 01:15:10.727033 8763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:10.728384 8746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:15:10.728595 8746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "31c653670dd34377a4ed273f88a14720"
format_stamp: "Formatted at 2026-08-06 01:15:10 on dist-test-slave-7nm7"
server_key: "58e9fc8cf976c9301808418e0ce22521"
server_key_iv: "23ca99f154bbf936eb4567c959f031a1"
server_key_version: "encryptionkey@0"
I20260806 01:15:10.728834 8746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:15:10.763481 8746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:15:10.764430 8746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:15:10.764782 8746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:15:10.765813 8746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:15:10.767441 8746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:15:10.767584 8746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:10.767717 8746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:15:10.767807 8746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:10.805924 8746 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:45157
I20260806 01:15:10.805994 8936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:45157 every 8 connection(s)
I20260806 01:15:10.807276 8746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:15:10.810195 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 8746
I20260806 01:15:10.810480 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:15:10.811113 26546 external_mini_cluster.cc:1428] Setting key 72c3d6a6d35ce31a32226ba426c80f0b
I20260806 01:15:10.814049 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:32885
--builtin_ntp_servers=127.25.236.148:38449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 01:15:10.821762 8940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:10.809915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:33737 (local address 127.25.236.190:32885)
0806 01:15:10.810442 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:10.810448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:10.812045 (+ 1597us) server_negotiation.cc:407] Connection header received
0806 01:15:10.813989 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:10.813994 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:10.814081 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:10.814249 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:10.815632 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:10.816409 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:10.818877 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:10.819328 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:10.819431 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:10.821015 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:10.821020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:10.821023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:10.821040 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:10.821082 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:10.821094 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:10.821158 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:10.821427 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 01:15:10.821504 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":269,"threads_started":1}
I20260806 01:15:10.823202 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:10.810746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:32885 (local address 127.25.236.130:33737)
0806 01:15:10.811885 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:10.811934 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:10.813725 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:10.814509 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:10.814523 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:10.814636 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:10.815440 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:10.815467 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:10.816583 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:10.816591 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:10.818732 (+ 2141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:10.818749 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:10.819551 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:10.819560 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:10.819876 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:10.820665 (+ 789us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:15:10.820670 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:10.820721 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:15:10.820816 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:15:10.821188 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:15:10.821200 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:15:10.821211 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 01:15:10.821383 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 01:15:10.821783 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":432,"threads_started":1}
I20260806 01:15:10.824654 8938 heartbeater.cc:347] Connected to a master server at 127.25.236.190:32885
I20260806 01:15:10.825026 8938 heartbeater.cc:464] Registering TS with master...
I20260806 01:15:10.826050 8938 heartbeater.cc:511] Master 127.25.236.190:32885 requested a full tablet report, sending...
I20260806 01:15:10.827847 8448 ts_manager.cc:194] Registered new tserver with Master: 31c653670dd34377a4ed273f88a14720 (127.25.236.130:45157)
I20260806 01:15:10.828747 8448 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.236.130:33737
W20260806 01:15:11.180140 8942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:15:11.180452 8942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:15:11.180541 8942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:15:11.188266 8942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:15:11.188385 8942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:15:11.188490 8942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:15:11.198416 8942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:38449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:32885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:15:11.199826 8942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:15:11.201785 8942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:15:11.215238 8950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:11.215243 8952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:11.216040 8955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:15:11.216787 8942 server_base.cc:1034] running on GCE node
I20260806 01:15:11.217556 8942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:15:11.218701 8942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:15:11.219937 8942 hybrid_clock.cc:648] HybridClock initialized: now 1785978911219866 us; error 58 us; skew 500 ppm
I20260806 01:15:11.220309 8942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:15:11.222846 8942 webserver.cc:460] Webserver started at http://127.25.236.131:37817/ using document root <none> and password file <none>
I20260806 01:15:11.223580 8942 fs_manager.cc:359] Metadata directory not provided
I20260806 01:15:11.223706 8942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:15:11.224050 8942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:15:11.227133 8942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "56ff9f2c395a43359b4b6421a4d8312d"
format_stamp: "Formatted at 2026-08-06 01:15:11 on dist-test-slave-7nm7"
server_key: "fb3e7b350c69d9fdb4cf524b584eba8e"
server_key_iv: "0eb4cd5433321a0d00b51a98482499dc"
server_key_version: "encryptionkey@0"
I20260806 01:15:11.227978 8942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "56ff9f2c395a43359b4b6421a4d8312d"
format_stamp: "Formatted at 2026-08-06 01:15:11 on dist-test-slave-7nm7"
server_key: "fb3e7b350c69d9fdb4cf524b584eba8e"
server_key_iv: "0eb4cd5433321a0d00b51a98482499dc"
server_key_version: "encryptionkey@0"
I20260806 01:15:11.233809 8942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:15:11.237679 8962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:11.239450 8942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:15:11.239655 8942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "56ff9f2c395a43359b4b6421a4d8312d"
format_stamp: "Formatted at 2026-08-06 01:15:11 on dist-test-slave-7nm7"
server_key: "fb3e7b350c69d9fdb4cf524b584eba8e"
server_key_iv: "0eb4cd5433321a0d00b51a98482499dc"
server_key_version: "encryptionkey@0"
I20260806 01:15:11.239905 8942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:15:11.289355 8942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:15:11.290627 8942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:15:11.290951 8942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:15:11.292084 8942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:15:11.293672 8942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:15:11.293807 8942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:11.293928 8942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:15:11.294014 8942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:11.301163 8740 heartbeater.cc:503] Master 127.25.236.190:32885 was elected leader, sending a full tablet report...
I20260806 01:15:11.333775 8942 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:36757
I20260806 01:15:11.333871 9137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:36757 every 8 connection(s)
I20260806 01:15:11.335227 8942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:15:11.339974 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 8942
I20260806 01:15:11.340168 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:15:11.340679 26546 external_mini_cluster.cc:1428] Setting key d114511f2643f3d79ee57861726490a4
I20260806 01:15:11.349558 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:11.338463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:35259 (local address 127.25.236.190:32885)
0806 01:15:11.338958 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:11.338963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:11.340625 (+ 1662us) server_negotiation.cc:407] Connection header received
0806 01:15:11.342738 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:11.342743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:11.342835 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:11.342955 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:11.344204 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:11.344982 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:11.346802 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:11.347227 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:11.347366 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:11.348678 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:11.348682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:11.348685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:11.348710 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:11.348751 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:11.348763 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:11.348860 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:11.349080 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 01:15:11.349181 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":276,"threads_started":1}
I20260806 01:15:11.350667 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:11.338969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:32885 (local address 127.25.236.131:35259)
0806 01:15:11.340449 (+ 1480us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:11.340492 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:11.342474 (+ 1982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:11.343228 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:11.343236 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:11.343319 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:11.344036 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:11.344054 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:11.345132 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:11.345140 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:11.346654 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:11.346667 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:11.347457 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:11.347465 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:11.347689 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:11.348374 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:15:11.348376 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:11.348414 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:15:11.348487 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:15:11.348862 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:15:11.348875 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:15:11.348885 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 01:15:11.349070 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 01:15:11.349504 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":421,"threads_started":1}
I20260806 01:15:11.352012 9138 heartbeater.cc:347] Connected to a master server at 127.25.236.190:32885
I20260806 01:15:11.352334 9138 heartbeater.cc:464] Registering TS with master...
I20260806 01:15:11.353086 9138 heartbeater.cc:511] Master 127.25.236.190:32885 requested a full tablet report, sending...
I20260806 01:15:11.354815 8448 ts_manager.cc:194] Registered new tserver with Master: 56ff9f2c395a43359b4b6421a4d8312d (127.25.236.131:36757)
I20260806 01:15:11.355942 8448 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.236.131:35259
I20260806 01:15:11.356166 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 01:15:11.363284 9140 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:11.361462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48184 (local address 127.25.236.190:32885)
0806 01:15:11.361658 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:11.361662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:11.362303 (+ 641us) server_negotiation.cc:407] Connection header received
0806 01:15:11.362464 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:11.362468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:11.362529 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:11.362608 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:11.362973 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48184: BlockingRecv error: recv got EOF from 127.0.0.1:48184 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:15:11.364159 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 8544
I20260806 01:15:11.374648 9156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:11.365156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60648 (local address 127.25.236.129:44357)
0806 01:15:11.366123 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:11.366130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:11.366157 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:15:11.366222 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:11.366229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:11.366612 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:11.366793 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:11.367562 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:11.369426 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:11.370971 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:11.371573 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:11.371756 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:11.372955 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:11.372964 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:11.372977 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:11.373640 (+ 663us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:11.373694 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:11.373715 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:11.373844 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:11.373999 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 01:15:11.374137 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":410,"threads_started":1}
I20260806 01:15:11.375999 8733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:15:11.462726 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 8544
I20260806 01:15:11.486735 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 8746
I20260806 01:15:11.500231 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:11.487789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39966 (local address 127.25.236.130:45157)
0806 01:15:11.488354 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:11.488361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:11.488390 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:15:11.488450 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:11.488457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:11.488683 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:11.488813 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:11.489226 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:11.490318 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:11.492002 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:11.492605 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:11.492714 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:11.498751 (+ 6037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:11.498759 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:11.498773 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:11.499445 (+ 672us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:11.499495 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:11.499515 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:11.499625 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:11.499776 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 01:15:11.499895 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":281,"threads_started":1}
I20260806 01:15:11.501854 8928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:15:11.587658 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 8746
I20260806 01:15:11.622530 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 8942
I20260806 01:15:11.633831 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:11.623832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32968 (local address 127.25.236.131:36757)
0806 01:15:11.624567 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:11.624574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:11.624607 (+ 33us) server_negotiation.cc:407] Connection header received
0806 01:15:11.624673 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:11.624679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:11.624909 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:11.625112 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:11.625586 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:11.627153 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:11.628910 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:11.630026 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:11.630485 (+ 459us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:11.632126 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:11.632136 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:11.632159 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:11.632836 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:11.632916 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:11.632939 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:11.633076 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:11.633267 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 01:15:11.633434 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":421,"threads_started":1}
I20260806 01:15:11.635279 9128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:15:11.727066 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 8942
I20260806 01:15:11.754925 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 8407
I20260806 01:15:11.756273 8498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:15:11.867233 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 8407
2026-08-06T01:15:11Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (3871 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T01:15:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:15:11Z Disabled control of system clock
I20260806 01:15:11.962077 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:36911
--webserver_interface=127.25.236.190
--webserver_port=0
--builtin_ntp_servers=127.25.236.148:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:36911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 01:15:12.335760 9174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:15:12.336799 9174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:15:12.337082 9174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:15:12.344970 9174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:15:12.345119 9174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:15:12.345160 9174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:15:12.345315 9174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:15:12.345403 9174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:15:12.355058 9174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:36911
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:36911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:15:12.356714 9174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:15:12.358649 9174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:15:12.372756 9182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:12.373418 9181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:12.375047 9187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:15:12.375602 9174 server_base.cc:1034] running on GCE node
I20260806 01:15:12.377390 9174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:15:12.380661 9174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:15:12.381903 9174 hybrid_clock.cc:648] HybridClock initialized: now 1785978912381833 us; error 71 us; skew 500 ppm
I20260806 01:15:12.382423 9174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:15:12.387115 9174 webserver.cc:460] Webserver started at http://127.25.236.190:33409/ using document root <none> and password file <none>
I20260806 01:15:12.388233 9174 fs_manager.cc:359] Metadata directory not provided
I20260806 01:15:12.388399 9174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:15:12.388828 9174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:15:12.393179 9174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "bff877d03b014b32996671053d88d5a3"
format_stamp: "Formatted at 2026-08-06 01:15:12 on dist-test-slave-7nm7"
server_key: "cd96f9ab8a1faf2029ed919896979b86"
server_key_iv: "3b9c0505964066af72baf365a960063b"
server_key_version: "encryptionkey@0"
I20260806 01:15:12.394030 9174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "bff877d03b014b32996671053d88d5a3"
format_stamp: "Formatted at 2026-08-06 01:15:12 on dist-test-slave-7nm7"
server_key: "cd96f9ab8a1faf2029ed919896979b86"
server_key_iv: "3b9c0505964066af72baf365a960063b"
server_key_version: "encryptionkey@0"
I20260806 01:15:12.400080 9174 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:15:12.405699 9193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:12.407712 9174 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 01:15:12.407893 9174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "bff877d03b014b32996671053d88d5a3"
format_stamp: "Formatted at 2026-08-06 01:15:12 on dist-test-slave-7nm7"
server_key: "cd96f9ab8a1faf2029ed919896979b86"
server_key_iv: "3b9c0505964066af72baf365a960063b"
server_key_version: "encryptionkey@0"
I20260806 01:15:12.408164 9174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:15:12.418895 9174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:15:12.419960 9174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:15:12.420298 9174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:15:12.443322 9174 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:36911
I20260806 01:15:12.443351 9268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:36911 every 8 connection(s)
I20260806 01:15:12.445353 9174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:15:12.450739 9270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:15:12.454612 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 9174
I20260806 01:15:12.454850 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:15:12.455528 26546 external_mini_cluster.cc:1428] Setting key e7bcd381a035850a03c7bbb2bcbdb1ac
I20260806 01:15:12.465394 9270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3: Bootstrap starting.
I20260806 01:15:12.467754 9275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:12.457999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43136 (local address 127.25.236.190:36911)
0806 01:15:12.459364 (+ 1365us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:12.459381 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:12.459441 (+ 60us) server_negotiation.cc:407] Connection header received
0806 01:15:12.462764 (+ 3323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:12.462798 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:12.463391 (+ 593us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:12.463864 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:12.465508 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:12.465517 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:12.465533 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:12.465552 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:12.465632 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:12.465654 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:12.465903 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:12.466041 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 01:15:12.466352 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":558,"threads_started":1}
I20260806 01:15:12.469530 9270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3: Neither blocks nor log segments found. Creating new log.
I20260806 01:15:12.470767 9270 log.cc:824] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3: Log is configured to *not* fsync() on all Append() calls
I20260806 01:15:12.473711 9270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3: No bootstrap required, opened a new log
I20260806 01:15:12.482968 9270 raft_consensus.cc:348] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bff877d03b014b32996671053d88d5a3" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36911 } }
I20260806 01:15:12.483628 9270 raft_consensus.cc:374] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:15:12.483776 9270 raft_consensus.cc:729] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff877d03b014b32996671053d88d5a3, State: Initialized, Role: FOLLOWER
I20260806 01:15:12.484650 9270 consensus_queue.cc:260] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bff877d03b014b32996671053d88d5a3" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36911 } }
I20260806 01:15:12.484984 9270 raft_consensus.cc:388] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:15:12.485152 9270 raft_consensus.cc:482] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:15:12.485399 9270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:15:12.488843 9270 raft_consensus.cc:504] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bff877d03b014b32996671053d88d5a3" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36911 } }
I20260806 01:15:12.489552 9270 leader_election.cc:302] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bff877d03b014b32996671053d88d5a3; no voters:
I20260806 01:15:12.490315 9270 leader_election.cc:288] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:15:12.490538 9277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:15:12.491863 9277 raft_consensus.cc:686] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [term 1 LEADER]: Becoming Leader. State: Replica: bff877d03b014b32996671053d88d5a3, State: Running, Role: LEADER
I20260806 01:15:12.492713 9277 consensus_queue.cc:237] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bff877d03b014b32996671053d88d5a3" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36911 } }
I20260806 01:15:12.492897 9270 sys_catalog.cc:564] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:15:12.497175 9280 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bff877d03b014b32996671053d88d5a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bff877d03b014b32996671053d88d5a3" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36911 } } }
I20260806 01:15:12.497646 9280 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [sys.catalog]: This master's current role is: LEADER
I20260806 01:15:12.498411 9281 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bff877d03b014b32996671053d88d5a3. Latest consensus state: current_term: 1 leader_uuid: "bff877d03b014b32996671053d88d5a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bff877d03b014b32996671053d88d5a3" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36911 } } }
I20260806 01:15:12.498821 9281 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3 [sys.catalog]: This master's current role is: LEADER
I20260806 01:15:12.501189 9291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:15:12.506067 9291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:15:12.517923 9291 catalog_manager.cc:1347] Generated new cluster ID: e092a0e72ff74a6ebf9784106edc5fe9
I20260806 01:15:12.518074 9291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:15:12.541280 9291 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:15:12.541752 9291 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:15:12.553609 9291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bff877d03b014b32996671053d88d5a3: Generated new TSK 0
I20260806 01:15:12.554566 9291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:15:12.571342 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:36911
--builtin_ntp_servers=127.25.236.148:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 01:15:13.000597 9308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:15:13.000890 9308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:15:13.001044 9308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:15:13.009055 9308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:15:13.009172 9308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:15:13.009339 9308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:15:13.021289 9308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:36911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:15:13.023013 9308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:15:13.026182 9308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:15:13.040729 9315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:13.040793 9314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:13.040819 9317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:15:13.042203 9308 server_base.cc:1034] running on GCE node
I20260806 01:15:13.043154 9308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:15:13.044430 9308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:15:13.045874 9308 hybrid_clock.cc:648] HybridClock initialized: now 1785978913045790 us; error 72 us; skew 500 ppm
I20260806 01:15:13.046371 9308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:15:13.049309 9308 webserver.cc:460] Webserver started at http://127.25.236.129:44485/ using document root <none> and password file <none>
I20260806 01:15:13.050060 9308 fs_manager.cc:359] Metadata directory not provided
I20260806 01:15:13.050154 9308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:15:13.050551 9308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:15:13.053723 9308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "0d5abc4e82ab45e0adf14483bb2ff72e"
format_stamp: "Formatted at 2026-08-06 01:15:13 on dist-test-slave-7nm7"
server_key: "23f57743ccd76600a35dd5ee4d8f0796"
server_key_iv: "4153009f0d22d855e4d95edcd5b0a648"
server_key_version: "encryptionkey@0"
I20260806 01:15:13.054615 9308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "0d5abc4e82ab45e0adf14483bb2ff72e"
format_stamp: "Formatted at 2026-08-06 01:15:13 on dist-test-slave-7nm7"
server_key: "23f57743ccd76600a35dd5ee4d8f0796"
server_key_iv: "4153009f0d22d855e4d95edcd5b0a648"
server_key_version: "encryptionkey@0"
I20260806 01:15:13.060122 9308 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:15:13.063912 9327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:13.065462 9308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:15:13.065649 9308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "0d5abc4e82ab45e0adf14483bb2ff72e"
format_stamp: "Formatted at 2026-08-06 01:15:13 on dist-test-slave-7nm7"
server_key: "23f57743ccd76600a35dd5ee4d8f0796"
server_key_iv: "4153009f0d22d855e4d95edcd5b0a648"
server_key_version: "encryptionkey@0"
I20260806 01:15:13.065896 9308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:15:13.071890 9308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:15:13.072734 9308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:15:13.073020 9308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:15:13.074019 9308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:15:13.075500 9308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:15:13.075596 9308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:13.075738 9308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:15:13.075830 9308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:13.112782 9308 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:38227
I20260806 01:15:13.112838 9511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:38227 every 8 connection(s)
I20260806 01:15:13.114182 9308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:15:13.122922 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 9308
I20260806 01:15:13.123449 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:15:13.124630 26546 external_mini_cluster.cc:1428] Setting key 09df5d69e6fd4c2a8977ffc467a52dbc
I20260806 01:15:13.126626 9516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:13.117445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:51911 (local address 127.25.236.190:36911)
0806 01:15:13.117976 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:13.117981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:13.119726 (+ 1745us) server_negotiation.cc:407] Connection header received
0806 01:15:13.122582 (+ 2856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:13.122588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:13.122678 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:13.122793 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:13.125270 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:13.125275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:13.125279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:13.125297 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:13.125350 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:13.125372 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:13.125504 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:13.125909 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 01:15:13.125986 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":279,"threads_started":1}
I20260806 01:15:13.128122 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:13.118180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:36911 (local address 127.25.236.129:51911)
0806 01:15:13.119528 (+ 1348us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:13.119572 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:13.121953 (+ 2381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:13.123424 (+ 1471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:13.123442 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:13.123594 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:13.124653 (+ 1059us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:15:13.124660 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:13.124761 (+ 101us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:15:13.124957 (+ 196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:15:13.125516 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:15:13.125531 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:15:13.125542 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 01:15:13.125868 (+ 326us) client_negotiation.cc:240] Negotiation successful
0806 01:15:13.126299 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":415,"threads_started":1}
I20260806 01:15:13.128585 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:36911
--builtin_ntp_servers=127.25.236.148:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 01:15:13.130002 9513 heartbeater.cc:347] Connected to a master server at 127.25.236.190:36911
I20260806 01:15:13.130632 9513 heartbeater.cc:464] Registering TS with master...
I20260806 01:15:13.131415 9513 heartbeater.cc:511] Master 127.25.236.190:36911 requested a full tablet report, sending...
I20260806 01:15:13.133833 9218 ts_manager.cc:194] Registered new tserver with Master: 0d5abc4e82ab45e0adf14483bb2ff72e (127.25.236.129:38227)
W20260806 01:15:13.492134 9521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:15:13.492426 9521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:15:13.492517 9521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:15:13.500239 9521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:15:13.500356 9521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:15:13.500468 9521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:15:13.511071 9521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:36911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:15:13.512408 9521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:15:13.514256 9521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:15:13.528318 9539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:13.528466 9540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:13.528883 9543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:15:13.529374 9521 server_base.cc:1034] running on GCE node
I20260806 01:15:13.530021 9521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:15:13.531055 9521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:15:13.532264 9521 hybrid_clock.cc:648] HybridClock initialized: now 1785978913532202 us; error 59 us; skew 500 ppm
I20260806 01:15:13.532618 9521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:15:13.535075 9521 webserver.cc:460] Webserver started at http://127.25.236.130:45951/ using document root <none> and password file <none>
I20260806 01:15:13.535782 9521 fs_manager.cc:359] Metadata directory not provided
I20260806 01:15:13.535902 9521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:15:13.536252 9521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:15:13.541906 9521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "d88ff6d921284e05b5d3cc744b0c1203"
format_stamp: "Formatted at 2026-08-06 01:15:13 on dist-test-slave-7nm7"
server_key: "40815f799951d35365ad13bf137c4c99"
server_key_iv: "94e3daea05b2935268663765fcdacd89"
server_key_version: "encryptionkey@0"
I20260806 01:15:13.542788 9521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "d88ff6d921284e05b5d3cc744b0c1203"
format_stamp: "Formatted at 2026-08-06 01:15:13 on dist-test-slave-7nm7"
server_key: "40815f799951d35365ad13bf137c4c99"
server_key_iv: "94e3daea05b2935268663765fcdacd89"
server_key_version: "encryptionkey@0"
I20260806 01:15:13.548456 9521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:15:13.552361 9552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:13.553843 9521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:15:13.554035 9521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "d88ff6d921284e05b5d3cc744b0c1203"
format_stamp: "Formatted at 2026-08-06 01:15:13 on dist-test-slave-7nm7"
server_key: "40815f799951d35365ad13bf137c4c99"
server_key_iv: "94e3daea05b2935268663765fcdacd89"
server_key_version: "encryptionkey@0"
I20260806 01:15:13.554466 9521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:15:13.560323 9521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:15:13.561138 9521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:15:13.561425 9521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:15:13.562541 9521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:15:13.563966 9521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:15:13.564081 9521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:13.564191 9521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:15:13.564285 9521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:13.600957 9521 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:36963
I20260806 01:15:13.601059 9728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:36963 every 8 connection(s)
I20260806 01:15:13.602344 9521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:15:13.608577 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 9521
I20260806 01:15:13.608798 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:15:13.609236 26546 external_mini_cluster.cc:1428] Setting key 6aab7553b37bf9794f873995395666b3
I20260806 01:15:13.610905 9516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:13.605040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:51085 (local address 127.25.236.190:36911)
0806 01:15:13.605387 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:13.605392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:13.606827 (+ 1435us) server_negotiation.cc:407] Connection header received
0806 01:15:13.608547 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:13.608553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:13.608628 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:13.608744 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:13.610183 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:13.610189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:13.610194 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:13.610214 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:13.610278 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:13.610295 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:13.610438 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:13.610646 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 01:15:13.610697 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212}
I20260806 01:15:13.612022 9733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:13.605599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:36911 (local address 127.25.236.130:51085)
0806 01:15:13.606638 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:13.606684 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:13.608272 (+ 1588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:13.608984 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:13.608998 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:13.609083 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:13.609799 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:15:13.609802 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:13.609867 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:15:13.609981 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:15:13.610395 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:15:13.610421 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:15:13.610432 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 01:15:13.610727 (+ 295us) client_negotiation.cc:240] Negotiation successful
0806 01:15:13.610967 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":318,"threads_started":1}
I20260806 01:15:13.612394 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:36911
--builtin_ntp_servers=127.25.236.148:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 01:15:13.613286 9729 heartbeater.cc:347] Connected to a master server at 127.25.236.190:36911
I20260806 01:15:13.613601 9729 heartbeater.cc:464] Registering TS with master...
I20260806 01:15:13.614182 9729 heartbeater.cc:511] Master 127.25.236.190:36911 requested a full tablet report, sending...
I20260806 01:15:13.615650 9218 ts_manager.cc:194] Registered new tserver with Master: d88ff6d921284e05b5d3cc744b0c1203 (127.25.236.130:36963)
W20260806 01:15:13.985201 9734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:15:13.985697 9734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:15:13.985961 9734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:15:13.993817 9734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:15:13.993930 9734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:15:13.994117 9734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:15:14.005537 9734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:36911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:15:14.007055 9734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:15:14.008890 9734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:15:14.023633 9746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:14.023633 9747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:14.023713 9750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:15:14.024531 9734 server_base.cc:1034] running on GCE node
I20260806 01:15:14.025204 9734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:15:14.026278 9734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:15:14.027498 9734 hybrid_clock.cc:648] HybridClock initialized: now 1785978914027433 us; error 66 us; skew 500 ppm
I20260806 01:15:14.027855 9734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:15:14.030411 9734 webserver.cc:460] Webserver started at http://127.25.236.131:44539/ using document root <none> and password file <none>
I20260806 01:15:14.031139 9734 fs_manager.cc:359] Metadata directory not provided
I20260806 01:15:14.031266 9734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:15:14.031728 9734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:15:14.035073 9734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "7c48a99ae904451fa38ad1c6464b0420"
format_stamp: "Formatted at 2026-08-06 01:15:14 on dist-test-slave-7nm7"
server_key: "89ed292b03b7f746bd22de78be5232bb"
server_key_iv: "003c97d9fbd6df9f0055002f79c47dfc"
server_key_version: "encryptionkey@0"
I20260806 01:15:14.035902 9734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "7c48a99ae904451fa38ad1c6464b0420"
format_stamp: "Formatted at 2026-08-06 01:15:14 on dist-test-slave-7nm7"
server_key: "89ed292b03b7f746bd22de78be5232bb"
server_key_iv: "003c97d9fbd6df9f0055002f79c47dfc"
server_key_version: "encryptionkey@0"
I20260806 01:15:14.041574 9734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 01:15:14.045658 9759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:14.047379 9734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 01:15:14.047564 9734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "7c48a99ae904451fa38ad1c6464b0420"
format_stamp: "Formatted at 2026-08-06 01:15:14 on dist-test-slave-7nm7"
server_key: "89ed292b03b7f746bd22de78be5232bb"
server_key_iv: "003c97d9fbd6df9f0055002f79c47dfc"
server_key_version: "encryptionkey@0"
I20260806 01:15:14.047796 9734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:15:14.054059 9734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:15:14.055087 9734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:15:14.055385 9734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:15:14.056381 9734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:15:14.057888 9734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:15:14.057976 9734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:14.058107 9734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:15:14.058192 9734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:14.098443 9734 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:41757
I20260806 01:15:14.098495 9941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:41757 every 8 connection(s)
I20260806 01:15:14.099874 9734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:15:14.105960 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 9734
I20260806 01:15:14.106177 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:15:14.106642 26546 external_mini_cluster.cc:1428] Setting key a3c70301299ddd6c9708f45294781891
I20260806 01:15:14.110292 9516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:14.103290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:54611 (local address 127.25.236.190:36911)
0806 01:15:14.103507 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:14.103512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:14.105464 (+ 1952us) server_negotiation.cc:407] Connection header received
0806 01:15:14.107462 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:14.107468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:14.107546 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:14.107670 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:14.109450 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:14.109458 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:14.109467 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:14.109486 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:14.109519 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:14.109531 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:14.109658 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:14.109884 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 01:15:14.109959 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 01:15:14.111327 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:14.104101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:36911 (local address 127.25.236.131:54611)
0806 01:15:14.105312 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:14.105347 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:14.107123 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:14.107984 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:14.107997 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:14.108217 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:14.109133 (+ 916us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:15:14.109136 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:14.109181 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:15:14.109271 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:15:14.109635 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:15:14.109646 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:15:14.109654 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 01:15:14.109958 (+ 304us) client_negotiation.cc:240] Negotiation successful
0806 01:15:14.110273 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":395,"threads_started":1}
I20260806 01:15:14.112849 9942 heartbeater.cc:347] Connected to a master server at 127.25.236.190:36911
I20260806 01:15:14.113204 9942 heartbeater.cc:464] Registering TS with master...
I20260806 01:15:14.113963 9942 heartbeater.cc:511] Master 127.25.236.190:36911 requested a full tablet report, sending...
I20260806 01:15:14.115658 9218 ts_manager.cc:194] Registered new tserver with Master: 7c48a99ae904451fa38ad1c6464b0420 (127.25.236.131:41757)
I20260806 01:15:14.123384 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 01:15:14.129990 9516 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:15:14.128042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43150 (local address 127.25.236.190:36911)
0806 01:15:14.128447 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:14.128465 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:14.128895 (+ 430us) server_negotiation.cc:407] Connection header received
0806 01:15:14.129024 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:14.129028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:14.129195 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:14.129314 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:14.129678 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43150: BlockingRecv error: recv got EOF from 127.0.0.1:43150 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 01:15:14.131726 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 9308
I20260806 01:15:14.136067 9956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:14.133100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.25.236.129:38227)
0806 01:15:14.133713 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:14.133729 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:14.133754 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:15:14.133812 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:14.133819 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:14.134024 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:14.134197 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:14.135319 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:14.135325 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:14.135337 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:14.135349 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:14.135389 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:14.135402 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:14.135589 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:14.135657 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 01:15:14.135778 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":261,"threads_started":1}
I20260806 01:15:14.137243 9513 heartbeater.cc:503] Master 127.25.236.190:36911 was elected leader, sending a full tablet report...
I20260806 01:15:14.137739 9505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:15:14.225132 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 9308
I20260806 01:15:14.256852 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 9521
I20260806 01:15:14.261610 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:14.257938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51294 (local address 127.25.236.130:36963)
0806 01:15:14.258520 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:14.258528 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:14.258554 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:15:14.258757 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:14.258854 (+ 97us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:14.259141 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:14.259289 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:14.260278 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:14.260286 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:14.260300 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:14.260316 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:14.260369 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:14.260388 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:14.260503 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:14.261050 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 01:15:14.261259 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":240,"threads_started":1}
I20260806 01:15:14.263311 9722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:15:14.351608 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 9521
I20260806 01:15:14.378088 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 9734
I20260806 01:15:14.382902 9961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:14.379202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48934 (local address 127.25.236.131:41757)
0806 01:15:14.379819 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:14.379826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:14.379856 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:15:14.379927 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:14.379934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:14.380145 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:14.380321 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:14.381759 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:14.381768 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:14.381783 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:15:14.381810 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:14.381868 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:15:14.381888 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:14.382205 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:14.382346 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 01:15:14.382466 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":303,"threads_started":1}
I20260806 01:15:14.384588 9935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:15:14.476364 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 9734
I20260806 01:15:14.500772 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 9174
I20260806 01:15:14.502023 9260 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:15:14.609941 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 9174
2026-08-06T01:15:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2744 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:15:14 dist-test-slave-7nm7 krb5kdc[9967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:15:14 dist-test-slave-7nm7 krb5kdc[9967](info): set up 2 sockets
Aug 06 01:15:14 dist-test-slave-7nm7 krb5kdc[9967](info): commencing operation
krb5kdc: starting...
W20260806 01:15:16.006906 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.345s user 0.006s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:15:16 dist-test-slave-7nm7 krb5kdc[9967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:15:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:15:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:15:16.354077 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:34701
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:34701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:15:16.734794 9984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:15:16.735102 9984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:15:16.735201 9984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:15:16.743634 9984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:15:16.743839 9984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:15:16.743896 9984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:15:16.743978 9984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:15:16.744030 9984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:15:16.755400 9984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:34701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:34701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:15:16.757488 9984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:15:16.758911 9984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:15:16.772001 9992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:16.772100 9997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:16.772128 9993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:15:16.772800 9984 server_base.cc:1034] running on GCE node
I20260806 01:15:16.773727 9984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:15:16.775358 9984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:15:16.776639 9984 hybrid_clock.cc:648] HybridClock initialized: now 1785978916776617 us; error 130 us; skew 500 ppm
I20260806 01:15:16.777037 9984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:15:16 dist-test-slave-7nm7 krb5kdc[9967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978916, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:15:16.785770 9984 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:15:16.787660 9984 webserver.cc:460] Webserver started at http://127.25.236.190:46573/ using document root <none> and password file <none>
I20260806 01:15:16.788431 9984 fs_manager.cc:359] Metadata directory not provided
I20260806 01:15:16.788599 9984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:15:16.788931 9984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:15:16.793429 9984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "3bd2bdf4bd2149abba9274729a075594"
format_stamp: "Formatted at 2026-08-06 01:15:16 on dist-test-slave-7nm7"
server_key: "8d625cefe5a961d3cd9134310c84f0fa"
server_key_iv: "739d5c5c13dbfe0cbdf08189d1f1c2ec"
server_key_version: "encryptionkey@0"
I20260806 01:15:16.794373 9984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "3bd2bdf4bd2149abba9274729a075594"
format_stamp: "Formatted at 2026-08-06 01:15:16 on dist-test-slave-7nm7"
server_key: "8d625cefe5a961d3cd9134310c84f0fa"
server_key_iv: "739d5c5c13dbfe0cbdf08189d1f1c2ec"
server_key_version: "encryptionkey@0"
I20260806 01:15:16.801270 9984 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260806 01:15:16.805573 10008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:16.807727 9984 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260806 01:15:16.807945 9984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "3bd2bdf4bd2149abba9274729a075594"
format_stamp: "Formatted at 2026-08-06 01:15:16 on dist-test-slave-7nm7"
server_key: "8d625cefe5a961d3cd9134310c84f0fa"
server_key_iv: "739d5c5c13dbfe0cbdf08189d1f1c2ec"
server_key_version: "encryptionkey@0"
I20260806 01:15:16.808172 9984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:15:16.835250 9984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:15:16.841369 9984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:15:16.841719 9984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:15:16.866364 10092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:34701 every 8 connection(s)
I20260806 01:15:16.866372 9984 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:34701
I20260806 01:15:16.867952 9984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:15:16.872445 10093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:15:16.878441 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 9984
I20260806 01:15:16.878716 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:15:16.879599 26546 external_mini_cluster.cc:1428] Setting key a74876c5cf834bf9e7bb1e1b26aedad0
I20260806 01:15:16.884428 10093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594: Bootstrap starting.
I20260806 01:15:16.889205 10093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594: Neither blocks nor log segments found. Creating new log.
I20260806 01:15:16.891161 10093 log.cc:824] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594: Log is configured to *not* fsync() on all Append() calls
I20260806 01:15:16.895977 10093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594: No bootstrap required, opened a new log
Aug 06 01:15:16 dist-test-slave-7nm7 krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:15:16.905570 10093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd2bdf4bd2149abba9274729a075594" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34701 } }
I20260806 01:15:16.906034 10093 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:15:16.906152 10093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd2bdf4bd2149abba9274729a075594, State: Initialized, Role: FOLLOWER
I20260806 01:15:16.906822 10093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd2bdf4bd2149abba9274729a075594" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34701 } }
I20260806 01:15:16.907104 10093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:15:16.907356 10093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:15:16.907598 10093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:15:16.909694 10097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:16.881895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.25.236.190:34701)
0806 01:15:16.882816 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:16.882828 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:16.882870 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:15:16.885147 (+ 2277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:16.885172 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:16.885570 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:16.885907 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:16.888449 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:16.890082 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:16.892793 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:16.893676 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:16.893837 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:16.899539 (+ 5702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:16.899548 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:16.899566 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:15:16.899615 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:16.904099 (+ 4484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:16.905400 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:16.905410 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:16.905419 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:16.905655 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:16.906389 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:16.906393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:16.906396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:16.907146 (+ 750us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:16.907664 (+ 518us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:16.908072 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 01:15:16.908461 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":476,"threads_started":1}
I20260806 01:15:16.911624 10093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd2bdf4bd2149abba9274729a075594" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34701 } }
I20260806 01:15:16.912767 10093 leader_election.cc:302] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bd2bdf4bd2149abba9274729a075594; no voters:
I20260806 01:15:16.913800 10093 leader_election.cc:288] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:15:16.913913 10100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:15:16.914954 10100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [term 1 LEADER]: Becoming Leader. State: Replica: 3bd2bdf4bd2149abba9274729a075594, State: Running, Role: LEADER
I20260806 01:15:16.915750 10100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd2bdf4bd2149abba9274729a075594" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34701 } }
I20260806 01:15:16.917430 10093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:15:16.919575 10102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bd2bdf4bd2149abba9274729a075594. Latest consensus state: current_term: 1 leader_uuid: "3bd2bdf4bd2149abba9274729a075594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd2bdf4bd2149abba9274729a075594" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34701 } } }
I20260806 01:15:16.919955 10102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [sys.catalog]: This master's current role is: LEADER
I20260806 01:15:16.919588 10101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bd2bdf4bd2149abba9274729a075594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd2bdf4bd2149abba9274729a075594" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34701 } } }
I20260806 01:15:16.920224 10101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594 [sys.catalog]: This master's current role is: LEADER
I20260806 01:15:16.921341 10108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:15:16.929237 10108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:15:16.938700 10108 catalog_manager.cc:1347] Generated new cluster ID: 43e0017980ab48179c1e2a25ceee09f6
I20260806 01:15:16.938844 10108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:15:16.946878 10108 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:15:16.948235 10108 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:15:16.962917 10108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594: Generated new TSK 0
I20260806 01:15:16.963812 10108 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:15:17.054378 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34701
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:15:17.462460 10132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:15:17.462765 10132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:15:17.462863 10132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:15:17.470914 10132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:15:17.471041 10132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:15:17.471165 10132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:15:17.483289 10132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:15:17.484678 10132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:15:17.486088 10132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:15:17.501784 10143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:17.501959 10148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:17.502056 10144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:15:17.503328 10132 server_base.cc:1034] running on GCE node
I20260806 01:15:17.504084 10132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:15:17.505203 10132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:15:17.506361 10132 hybrid_clock.cc:648] HybridClock initialized: now 1785978917506309 us; error 33 us; skew 500 ppm
I20260806 01:15:17.506671 10132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:15:17 dist-test-slave-7nm7 krb5kdc[9967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978917, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:15:17.511281 10132 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:15:17.513298 10132 webserver.cc:460] Webserver started at http://127.25.236.129:38169/ using document root <none> and password file <none>
I20260806 01:15:17.514021 10132 fs_manager.cc:359] Metadata directory not provided
I20260806 01:15:17.514338 10132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:15:17.514695 10132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:15:17.518726 10132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "895dad9fd46b417e951d7f20caac9a0c"
format_stamp: "Formatted at 2026-08-06 01:15:17 on dist-test-slave-7nm7"
server_key: "7ec7f6aae49b249558ade2d2d0d4718a"
server_key_iv: "ae42ae3b3147e0486d8d89441f70d630"
server_key_version: "encryptionkey@0"
I20260806 01:15:17.519600 10132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "895dad9fd46b417e951d7f20caac9a0c"
format_stamp: "Formatted at 2026-08-06 01:15:17 on dist-test-slave-7nm7"
server_key: "7ec7f6aae49b249558ade2d2d0d4718a"
server_key_iv: "ae42ae3b3147e0486d8d89441f70d630"
server_key_version: "encryptionkey@0"
I20260806 01:15:17.525189 10132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:15:17.530071 10159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:17.531862 10132 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:15:17.532053 10132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "895dad9fd46b417e951d7f20caac9a0c"
format_stamp: "Formatted at 2026-08-06 01:15:17 on dist-test-slave-7nm7"
server_key: "7ec7f6aae49b249558ade2d2d0d4718a"
server_key_iv: "ae42ae3b3147e0486d8d89441f70d630"
server_key_version: "encryptionkey@0"
I20260806 01:15:17.532279 10132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:15:17.548110 10132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:15:17.553450 10132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:15:17.553795 10132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:15:17.554905 10132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:15:17.556344 10132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:15:17.556454 10132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:17.556610 10132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:15:17.556700 10132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:17.595110 10132 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:40087
I20260806 01:15:17.595192 10336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:40087 every 8 connection(s)
I20260806 01:15:17.596721 10132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:15:17.604277 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 10132
I20260806 01:15:17.604529 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:15:17.605410 26546 external_mini_cluster.cc:1428] Setting key 54eddc80ceb10ebf7287c8f8fafe5ba0
Aug 06 01:15:17 dist-test-slave-7nm7 krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978917, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:15:17.622203 10341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:17.599720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:49683 (local address 127.25.236.190:34701)
0806 01:15:17.600404 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:17.600413 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:17.601433 (+ 1020us) server_negotiation.cc:407] Connection header received
0806 01:15:17.603448 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:17.603453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:17.603532 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:17.603625 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:17.606097 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:17.606980 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:17.609461 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:17.609900 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:17.610030 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:17.615147 (+ 5117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:17.615153 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:17.615156 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:15:17.615192 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:17.618799 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:17.619715 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:17.619721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:17.619725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:17.619865 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:17.620328 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:17.620334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:17.620337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:17.620639 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:17.620802 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:17.621552 (+ 750us) server_negotiation.cc:299] Negotiation successful
0806 01:15:17.621794 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":465,"threads_started":1}
I20260806 01:15:17.623488 10342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:17.600458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34701 (local address 127.25.236.129:49683)
0806 01:15:17.601259 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:17.601305 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:17.603036 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:17.604021 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:17.604035 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:17.604919 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:17.605828 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:17.605854 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:17.607307 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:17.607313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:17.609277 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:17.609297 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:17.610171 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:17.610182 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:17.610558 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:17.611391 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:17.611425 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:17.614886 (+ 3461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:15:17.618980 (+ 4094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:15:17.618991 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:15:17.619017 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:15:17.619570 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:15:17.619985 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:15:17.619990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:15:17.619993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:15:17.620206 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:15:17.620776 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:15:17.620785 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:15:17.621176 (+ 391us) client_negotiation.cc:769] Sending connection context
0806 01:15:17.621459 (+ 283us) client_negotiation.cc:240] Negotiation successful
0806 01:15:17.622050 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":298,"threads_started":1}
I20260806 01:15:17.625200 10338 heartbeater.cc:347] Connected to a master server at 127.25.236.190:34701
I20260806 01:15:17.625553 10338 heartbeater.cc:464] Registering TS with master...
I20260806 01:15:17.626406 10338 heartbeater.cc:511] Master 127.25.236.190:34701 requested a full tablet report, sending...
I20260806 01:15:17.629498 10033 ts_manager.cc:194] Registered new tserver with Master: 895dad9fd46b417e951d7f20caac9a0c (127.25.236.129:40087)
I20260806 01:15:17.631592 10033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:49683
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:15:17.683072 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34701
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:15:18.065212 10346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:15:18.065541 10346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:15:18.065716 10346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:15:18.073760 10346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:15:18.073899 10346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:15:18.074085 10346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:15:18.087023 10346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:15:18.089148 10346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:15:18.090829 10346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:15:18.106869 10353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:18.106823 10356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:18.106915 10354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:15:18.108630 10346 server_base.cc:1034] running on GCE node
I20260806 01:15:18.109367 10346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:15:18.110613 10346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:15:18.112057 10346 hybrid_clock.cc:648] HybridClock initialized: now 1785978918111957 us; error 59 us; skew 500 ppm
I20260806 01:15:18.112440 10346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:15:18 dist-test-slave-7nm7 krb5kdc[9967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978918, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:15:18.116355 10346 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:15:18.118446 10346 webserver.cc:460] Webserver started at http://127.25.236.130:42953/ using document root <none> and password file <none>
I20260806 01:15:18.119263 10346 fs_manager.cc:359] Metadata directory not provided
I20260806 01:15:18.119378 10346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:15:18.119696 10346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:15:18.122942 10346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "cce38cc01e854b5582c4c96da682a7ca"
format_stamp: "Formatted at 2026-08-06 01:15:18 on dist-test-slave-7nm7"
server_key: "4ed7fb35cf40aedda663ca2852cdca2a"
server_key_iv: "893b710c64dc92ffac4a8ce34c612338"
server_key_version: "encryptionkey@0"
I20260806 01:15:18.123826 10346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "cce38cc01e854b5582c4c96da682a7ca"
format_stamp: "Formatted at 2026-08-06 01:15:18 on dist-test-slave-7nm7"
server_key: "4ed7fb35cf40aedda663ca2852cdca2a"
server_key_iv: "893b710c64dc92ffac4a8ce34c612338"
server_key_version: "encryptionkey@0"
I20260806 01:15:18.129856 10346 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 01:15:18.133934 10371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:18.135924 10346 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260806 01:15:18.136147 10346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "cce38cc01e854b5582c4c96da682a7ca"
format_stamp: "Formatted at 2026-08-06 01:15:18 on dist-test-slave-7nm7"
server_key: "4ed7fb35cf40aedda663ca2852cdca2a"
server_key_iv: "893b710c64dc92ffac4a8ce34c612338"
server_key_version: "encryptionkey@0"
I20260806 01:15:18.136366 10346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:15:18.151934 10346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:15:18.156527 10346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:15:18.156867 10346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:15:18.158143 10346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:15:18.159794 10346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:15:18.159873 10346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:18.159958 10346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:15:18.160012 10346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:18.200713 10346 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:34877
I20260806 01:15:18.200774 10550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:34877 every 8 connection(s)
I20260806 01:15:18.202111 10346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:15:18.211815 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 10346
I20260806 01:15:18.212059 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:15:18.212633 26546 external_mini_cluster.cc:1428] Setting key 64fdd11fe56a84f78c49e00278e7e000
Aug 06 01:15:18 dist-test-slave-7nm7 krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978918, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:15:18.227782 10553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:18.204923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:45169 (local address 127.25.236.190:34701)
0806 01:15:18.205494 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:18.205502 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:18.206433 (+ 931us) server_negotiation.cc:407] Connection header received
0806 01:15:18.208468 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:18.208472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:18.208540 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:18.208658 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:18.210424 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.211594 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.213803 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.214478 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.214635 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:18.219086 (+ 4451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:18.219174 (+ 88us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:18.219180 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:15:18.219242 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:18.223288 (+ 4046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:18.224484 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:18.224491 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:18.224497 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:18.224666 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:18.225207 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:18.225214 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:18.225217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:18.225564 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:18.225736 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:18.227070 (+ 1334us) server_negotiation.cc:299] Negotiation successful
0806 01:15:18.227367 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":352,"threads_started":1}
I20260806 01:15:18.228837 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:18.205467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34701 (local address 127.25.236.130:45169)
0806 01:15:18.206274 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:18.206317 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:18.208190 (+ 1873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:18.208864 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:18.208872 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:18.209394 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:18.209959 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:18.209976 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.211782 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.211790 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:18.213634 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:18.213647 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.214760 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.214772 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:18.215084 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:18.215866 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:18.215891 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:18.218819 (+ 2928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:15:18.223571 (+ 4752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:15:18.223583 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:15:18.223605 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:15:18.224324 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:15:18.224829 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:15:18.224834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:15:18.224838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:15:18.225076 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:15:18.225714 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:15:18.225725 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:15:18.226207 (+ 482us) client_negotiation.cc:769] Sending connection context
0806 01:15:18.226911 (+ 704us) client_negotiation.cc:240] Negotiation successful
0806 01:15:18.227609 (+ 698us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":300,"threads_started":1}
I20260806 01:15:18.230866 10551 heartbeater.cc:347] Connected to a master server at 127.25.236.190:34701
I20260806 01:15:18.231242 10551 heartbeater.cc:464] Registering TS with master...
I20260806 01:15:18.232328 10551 heartbeater.cc:511] Master 127.25.236.190:34701 requested a full tablet report, sending...
I20260806 01:15:18.234825 10033 ts_manager.cc:194] Registered new tserver with Master: cce38cc01e854b5582c4c96da682a7ca (127.25.236.130:34877)
I20260806 01:15:18.236752 10033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:45169
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:15:18.291060 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34701
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:15:18.634950 10338 heartbeater.cc:503] Master 127.25.236.190:34701 was elected leader, sending a full tablet report...
W20260806 01:15:18.690836 10561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:15:18.691146 10561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:15:18.691253 10561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:15:18.700457 10561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:15:18.700588 10561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:15:18.700712 10561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:15:18.712165 10561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:15:18.713778 10561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:15:18.715163 10561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:15:18.729918 10573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:18.730284 10576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:15:18.730902 10570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:15:18.731387 10561 server_base.cc:1034] running on GCE node
I20260806 01:15:18.732131 10561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:15:18.733163 10561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:15:18.734467 10561 hybrid_clock.cc:648] HybridClock initialized: now 1785978918734422 us; error 38 us; skew 500 ppm
I20260806 01:15:18.734882 10561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:15:18 dist-test-slave-7nm7 krb5kdc[9967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978918, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:15:18.739283 10561 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:15:18.741554 10561 webserver.cc:460] Webserver started at http://127.25.236.131:38081/ using document root <none> and password file <none>
I20260806 01:15:18.742391 10561 fs_manager.cc:359] Metadata directory not provided
I20260806 01:15:18.742548 10561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:15:18.742872 10561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:15:18.746143 10561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "5d02e77b2be6476e891555c8cd2f5d17"
format_stamp: "Formatted at 2026-08-06 01:15:18 on dist-test-slave-7nm7"
server_key: "141f55013219b16dedb362baee05a78d"
server_key_iv: "865dd0c154a094d6117a56079fca53f3"
server_key_version: "encryptionkey@0"
I20260806 01:15:18.747083 10561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "5d02e77b2be6476e891555c8cd2f5d17"
format_stamp: "Formatted at 2026-08-06 01:15:18 on dist-test-slave-7nm7"
server_key: "141f55013219b16dedb362baee05a78d"
server_key_iv: "865dd0c154a094d6117a56079fca53f3"
server_key_version: "encryptionkey@0"
I20260806 01:15:18.753396 10561 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 01:15:18.757875 10584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:18.760248 10561 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 01:15:18.760632 10561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "5d02e77b2be6476e891555c8cd2f5d17"
format_stamp: "Formatted at 2026-08-06 01:15:18 on dist-test-slave-7nm7"
server_key: "141f55013219b16dedb362baee05a78d"
server_key_iv: "865dd0c154a094d6117a56079fca53f3"
server_key_version: "encryptionkey@0"
I20260806 01:15:18.760926 10561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:15:18.785037 10561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:15:18.791311 10561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:15:18.791712 10561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:15:18.792820 10561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:15:18.794404 10561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:15:18.794509 10561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:18.794680 10561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:15:18.794759 10561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:15:18.841886 10561 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:45755
I20260806 01:15:18.841956 10760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:45755 every 8 connection(s)
I20260806 01:15:18.843693 10561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:15:18.846478 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 10561
I20260806 01:15:18.846726 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:15:18.847230 26546 external_mini_cluster.cc:1428] Setting key 3e357f2b18339b47c7994890c42f8da7
Aug 06 01:15:18 dist-test-slave-7nm7 krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978918, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:15:18.867950 10765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:18.846733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:41635 (local address 127.25.236.190:34701)
0806 01:15:18.847419 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:18.847424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:18.848385 (+ 961us) server_negotiation.cc:407] Connection header received
0806 01:15:18.850606 (+ 2221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:18.850611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:18.850727 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:18.850848 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:18.852432 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.853198 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.854805 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.855222 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.855361 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:18.859366 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:18.859372 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:18.859376 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:15:18.859417 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:18.863870 (+ 4453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:18.864904 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:18.864908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:18.864910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:18.865139 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:18.865853 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:18.865858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:18.865862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:18.866280 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:18.866455 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:18.867293 (+ 838us) server_negotiation.cc:299] Negotiation successful
0806 01:15:18.867545 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":281,"threads_started":1}
I20260806 01:15:18.869727 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:18.847311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34701 (local address 127.25.236.131:41635)
0806 01:15:18.848232 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:18.848271 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:18.850345 (+ 2074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:18.851179 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:18.851187 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:18.851714 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:15:18.852261 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:18.852277 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.853357 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.853363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:18.854649 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:18.854673 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.855465 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.855473 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:18.855699 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:18.856480 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:15:18.856515 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:15:18.859077 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:15:18.864048 (+ 4971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:15:18.864059 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:15:18.864079 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:15:18.864658 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:15:18.865298 (+ 640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:15:18.865302 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:15:18.865305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:15:18.865703 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:15:18.866524 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:15:18.866530 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:15:18.866899 (+ 369us) client_negotiation.cc:769] Sending connection context
0806 01:15:18.867139 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 01:15:18.867949 (+ 810us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":376,"threads_started":1}
I20260806 01:15:18.871953 10761 heartbeater.cc:347] Connected to a master server at 127.25.236.190:34701
I20260806 01:15:18.872660 10761 heartbeater.cc:464] Registering TS with master...
I20260806 01:15:18.873833 10761 heartbeater.cc:511] Master 127.25.236.190:34701 requested a full tablet report, sending...
I20260806 01:15:18.876073 10033 ts_manager.cc:194] Registered new tserver with Master: 5d02e77b2be6476e891555c8cd2f5d17 (127.25.236.131:45755)
I20260806 01:15:18.877686 10033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:41635
I20260806 01:15:18.877921 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:15:18.897318 10765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:18.882870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41432 (local address 127.25.236.190:34701)
0806 01:15:18.883056 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:18.883060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:18.883781 (+ 721us) server_negotiation.cc:407] Connection header received
0806 01:15:18.883995 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:18.884006 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:18.884088 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:18.884200 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:18.885685 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.886480 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.887724 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.888196 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.888360 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:18.891249 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:18.891254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:18.891257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:15:18.891289 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:18.894065 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:18.895088 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:18.895097 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:18.895103 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:18.895250 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:18.895664 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:18.895670 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:18.895673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:18.896017 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:18.896367 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:18.896738 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 01:15:18.896996 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:15:18.902326 10033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:15:18.910324 10033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:15:18.937793 10786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:18.925224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:45755 (local address 127.0.0.1:51630)
0806 01:15:18.928423 (+ 3199us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:18.928447 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:18.928597 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:18.929482 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:18.929494 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:18.929541 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:15:18.930208 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:18.930248 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.932256 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.932260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:18.934706 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:18.934725 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.936703 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.936712 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:18.936931 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:18.936985 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 01:15:18.937199 (+ 214us) client_negotiation.cc:240] Negotiation successful
0806 01:15:18.937403 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3033,"thread_start_us":4743,"threads_started":1}
I20260806 01:15:18.938423 10789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:18.925904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51630 (local address 127.25.236.131:45755)
0806 01:15:18.927380 (+ 1476us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:18.927392 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:18.928719 (+ 1327us) server_negotiation.cc:407] Connection header received
0806 01:15:18.928853 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:18.928866 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:18.929184 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:18.929434 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:15:18.930492 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.932088 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.934888 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.936454 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.936664 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:18.936849 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:18.937230 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 01:15:18.937661 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1254,"thread_start_us":1090,"threads_started":1}
I20260806 01:15:18.939502 10783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:18.924950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:40087 (local address 127.0.0.1:32778)
0806 01:15:18.926403 (+ 1453us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:18.926460 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:18.926659 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:18.931347 (+ 4688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:18.931353 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:18.931383 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:15:18.931830 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:18.931842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.933550 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.933558 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:18.935710 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:18.935726 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.938164 (+ 2438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.938172 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:18.938665 (+ 493us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:18.938717 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 01:15:18.938961 (+ 244us) client_negotiation.cc:240] Negotiation successful
0806 01:15:18.939159 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1119,"mutex_wait_us":1,"thread_start_us":945,"threads_started":1}
I20260806 01:15:18.940647 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:18.925932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35260 (local address 127.25.236.130:34877)
0806 01:15:18.927869 (+ 1937us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:18.927879 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:18.930524 (+ 2645us) server_negotiation.cc:407] Connection header received
0806 01:15:18.931073 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:18.931081 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:18.931338 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:18.931502 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:15:18.932561 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.934134 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.936783 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.938747 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.939038 (+ 291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:18.939377 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:18.940093 (+ 716us) server_negotiation.cc:299] Negotiation successful
0806 01:15:18.940286 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1759,"thread_start_us":1645,"threads_started":1}
I20260806 01:15:18.941561 10785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:18.925645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:34877 (local address 127.0.0.1:35260)
0806 01:15:18.930414 (+ 4769us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:18.930433 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:18.930631 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:18.931816 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:18.931820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:18.931936 (+ 116us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:15:18.932361 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:18.932384 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.934329 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.934341 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:18.936574 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:18.936592 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.939626 (+ 3034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.939634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:18.939803 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:18.939837 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:15:18.940992 (+ 1155us) client_negotiation.cc:240] Negotiation successful
0806 01:15:18.941200 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4464,"thread_start_us":2436,"threads_started":1}
I20260806 01:15:18.942057 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:18.925178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32778 (local address 127.25.236.129:40087)
0806 01:15:18.930684 (+ 5506us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:18.930694 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:18.930722 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:15:18.930813 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:18.930819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:18.931122 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:18.932029 (+ 907us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:15:18.932324 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.933415 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.935900 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:18.937867 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:18.940414 (+ 2547us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:18.940791 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:18.940864 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 01:15:18.941263 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5333,"thread_start_us":5166,"threads_started":1}
I20260806 01:15:18.944525 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 099e92acc3b94b2b913ba35ac3fe5aae (DEFAULT_TABLE table=test-table [id=bd85b34dea05457399911404b956bf02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:15:18.946933 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099e92acc3b94b2b913ba35ac3fe5aae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:15:18.946580 10446 tablet_service.cc:1467] Processing CreateTablet for tablet 099e92acc3b94b2b913ba35ac3fe5aae (DEFAULT_TABLE table=test-table [id=bd85b34dea05457399911404b956bf02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:15:18.946568 10234 tablet_service.cc:1467] Processing CreateTablet for tablet 099e92acc3b94b2b913ba35ac3fe5aae (DEFAULT_TABLE table=test-table [id=bd85b34dea05457399911404b956bf02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:15:18.948228 10446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099e92acc3b94b2b913ba35ac3fe5aae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:15:18.948709 10234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099e92acc3b94b2b913ba35ac3fe5aae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:15:18.959714 10790 tablet_bootstrap.cc:492] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17: Bootstrap starting.
I20260806 01:15:18.964490 10790 tablet_bootstrap.cc:654] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17: Neither blocks nor log segments found. Creating new log.
I20260806 01:15:18.964749 10791 tablet_bootstrap.cc:492] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca: Bootstrap starting.
I20260806 01:15:18.966454 10790 log.cc:824] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17: Log is configured to *not* fsync() on all Append() calls
I20260806 01:15:18.967075 10796 tablet_bootstrap.cc:492] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c: Bootstrap starting.
I20260806 01:15:18.969532 10791 tablet_bootstrap.cc:654] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca: Neither blocks nor log segments found. Creating new log.
I20260806 01:15:18.970104 10796 tablet_bootstrap.cc:654] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c: Neither blocks nor log segments found. Creating new log.
I20260806 01:15:18.971361 10791 log.cc:824] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca: Log is configured to *not* fsync() on all Append() calls
I20260806 01:15:18.971470 10796 log.cc:824] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:15:18.974656 10796 tablet_bootstrap.cc:492] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c: No bootstrap required, opened a new log
I20260806 01:15:18.974974 10796 ts_tablet_manager.cc:1397] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 01:15:18.975816 10790 tablet_bootstrap.cc:492] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17: No bootstrap required, opened a new log
I20260806 01:15:18.976099 10790 ts_tablet_manager.cc:1397] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20260806 01:15:18.976271 10791 tablet_bootstrap.cc:492] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca: No bootstrap required, opened a new log
I20260806 01:15:18.976560 10791 ts_tablet_manager.cc:1397] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260806 01:15:18.985504 10790 raft_consensus.cc:348] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895dad9fd46b417e951d7f20caac9a0c" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 40087 } } peers { permanent_uuid: "5d02e77b2be6476e891555c8cd2f5d17" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 45755 } } peers { permanent_uuid: "cce38cc01e854b5582c4c96da682a7ca" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 34877 } }
I20260806 01:15:18.985999 10790 raft_consensus.cc:374] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:15:18.986150 10790 raft_consensus.cc:729] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d02e77b2be6476e891555c8cd2f5d17, State: Initialized, Role: FOLLOWER
I20260806 01:15:18.986627 10796 raft_consensus.cc:348] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895dad9fd46b417e951d7f20caac9a0c" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 40087 } } peers { permanent_uuid: "5d02e77b2be6476e891555c8cd2f5d17" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 45755 } } peers { permanent_uuid: "cce38cc01e854b5582c4c96da682a7ca" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 34877 } }
I20260806 01:15:18.986874 10790 consensus_queue.cc:260] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895dad9fd46b417e951d7f20caac9a0c" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 40087 } } peers { permanent_uuid: "5d02e77b2be6476e891555c8cd2f5d17" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 45755 } } peers { permanent_uuid: "cce38cc01e854b5582c4c96da682a7ca" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 34877 } }
I20260806 01:15:18.987077 10796 raft_consensus.cc:374] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:15:18.987353 10796 raft_consensus.cc:729] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 895dad9fd46b417e951d7f20caac9a0c, State: Initialized, Role: FOLLOWER
I20260806 01:15:18.987947 10791 raft_consensus.cc:348] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895dad9fd46b417e951d7f20caac9a0c" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 40087 } } peers { permanent_uuid: "5d02e77b2be6476e891555c8cd2f5d17" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 45755 } } peers { permanent_uuid: "cce38cc01e854b5582c4c96da682a7ca" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 34877 } }
I20260806 01:15:18.988370 10761 heartbeater.cc:503] Master 127.25.236.190:34701 was elected leader, sending a full tablet report...
I20260806 01:15:18.988410 10791 raft_consensus.cc:374] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:15:18.988555 10791 raft_consensus.cc:729] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cce38cc01e854b5582c4c96da682a7ca, State: Initialized, Role: FOLLOWER
I20260806 01:15:18.989114 10790 ts_tablet_manager.cc:1428] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260806 01:15:18.989172 10791 consensus_queue.cc:260] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895dad9fd46b417e951d7f20caac9a0c" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 40087 } } peers { permanent_uuid: "5d02e77b2be6476e891555c8cd2f5d17" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 45755 } } peers { permanent_uuid: "cce38cc01e854b5582c4c96da682a7ca" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 34877 } }
I20260806 01:15:18.989436 10796 consensus_queue.cc:260] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895dad9fd46b417e951d7f20caac9a0c" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 40087 } } peers { permanent_uuid: "5d02e77b2be6476e891555c8cd2f5d17" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 45755 } } peers { permanent_uuid: "cce38cc01e854b5582c4c96da682a7ca" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 34877 } }
I20260806 01:15:18.990497 10551 heartbeater.cc:503] Master 127.25.236.190:34701 was elected leader, sending a full tablet report...
I20260806 01:15:18.991055 10791 ts_tablet_manager.cc:1428] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260806 01:15:18.996917 10796 ts_tablet_manager.cc:1428] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c: Time spent starting tablet: real 0.022s user 0.012s sys 0.008s
I20260806 01:15:19.016234 10800 raft_consensus.cc:482] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:15:19.016604 10800 raft_consensus.cc:504] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895dad9fd46b417e951d7f20caac9a0c" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 40087 } } peers { permanent_uuid: "5d02e77b2be6476e891555c8cd2f5d17" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 45755 } } peers { permanent_uuid: "cce38cc01e854b5582c4c96da682a7ca" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 34877 } }
I20260806 01:15:19.018487 10800 leader_election.cc:288] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 895dad9fd46b417e951d7f20caac9a0c (127.25.236.129:40087), cce38cc01e854b5582c4c96da682a7ca (127.25.236.130:34877)
I20260806 01:15:19.026022 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:19.018993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:58103 (local address 127.25.236.129:40087)
0806 01:15:19.019215 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:19.019222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:19.019437 (+ 215us) server_negotiation.cc:407] Connection header received
0806 01:15:19.019612 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:19.019622 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:19.019747 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:19.020024 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:15:19.020576 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.021619 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.023957 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.025102 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.025283 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:19.025365 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:19.025669 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 01:15:19.025804 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:15:19.026105 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:19.018838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:40087 (local address 127.25.236.131:58103)
0806 01:15:19.019246 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:19.019268 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:19.019420 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:19.020050 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:19.020055 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:19.020097 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:15:19.020383 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:19.020392 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.021777 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.021781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:19.023790 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:19.023810 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.025306 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.025311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:19.025487 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:19.025513 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 01:15:19.025600 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 01:15:19.025782 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260806 01:15:19.026572 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:19.019024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:41835 (local address 127.25.236.130:34877)
0806 01:15:19.019440 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:19.019446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:19.020373 (+ 927us) server_negotiation.cc:407] Connection header received
0806 01:15:19.020544 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:19.020549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:19.020628 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:19.020742 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:15:19.021241 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.022087 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.024385 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.025545 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.025740 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:19.025804 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:19.026175 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 01:15:19.026335 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259}
I20260806 01:15:19.026681 10804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:19.018958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:34877 (local address 127.25.236.131:41835)
0806 01:15:19.020231 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0806 01:15:19.020251 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:15:19.020392 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:15:19.020760 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:15:19.020763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:15:19.020782 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:15:19.021083 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:19.021090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.022287 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.022292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:19.024210 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:15:19.024220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.025806 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.025812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:15:19.025987 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:19.026015 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 01:15:19.026136 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 01:15:19.026290 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1104,"thread_start_us":726,"threads_started":1}
I20260806 01:15:19.027616 10268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099e92acc3b94b2b913ba35ac3fe5aae" candidate_uuid: "5d02e77b2be6476e891555c8cd2f5d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "895dad9fd46b417e951d7f20caac9a0c" is_pre_election: true
I20260806 01:15:19.027874 10474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099e92acc3b94b2b913ba35ac3fe5aae" candidate_uuid: "5d02e77b2be6476e891555c8cd2f5d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cce38cc01e854b5582c4c96da682a7ca" is_pre_election: true
I20260806 01:15:19.028223 10268 raft_consensus.cc:2454] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d02e77b2be6476e891555c8cd2f5d17 in term 0.
I20260806 01:15:19.028365 10474 raft_consensus.cc:2454] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d02e77b2be6476e891555c8cd2f5d17 in term 0.
I20260806 01:15:19.028992 10588 leader_election.cc:302] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5d02e77b2be6476e891555c8cd2f5d17, 895dad9fd46b417e951d7f20caac9a0c, cce38cc01e854b5582c4c96da682a7ca; no voters:
I20260806 01:15:19.029549 10800 raft_consensus.cc:2781] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:15:19.029747 10800 raft_consensus.cc:482] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:15:19.029868 10800 raft_consensus.cc:3037] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:15:19.032495 10800 raft_consensus.cc:504] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895dad9fd46b417e951d7f20caac9a0c" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 40087 } } peers { permanent_uuid: "5d02e77b2be6476e891555c8cd2f5d17" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 45755 } } peers { permanent_uuid: "cce38cc01e854b5582c4c96da682a7ca" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 34877 } }
I20260806 01:15:19.033272 10800 leader_election.cc:288] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [CANDIDATE]: Term 1 election: Requested vote from peers 895dad9fd46b417e951d7f20caac9a0c (127.25.236.129:40087), cce38cc01e854b5582c4c96da682a7ca (127.25.236.130:34877)
I20260806 01:15:19.033741 10268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099e92acc3b94b2b913ba35ac3fe5aae" candidate_uuid: "5d02e77b2be6476e891555c8cd2f5d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "895dad9fd46b417e951d7f20caac9a0c"
I20260806 01:15:19.034018 10268 raft_consensus.cc:3037] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:15:19.034097 10474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099e92acc3b94b2b913ba35ac3fe5aae" candidate_uuid: "5d02e77b2be6476e891555c8cd2f5d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cce38cc01e854b5582c4c96da682a7ca"
I20260806 01:15:19.034432 10474 raft_consensus.cc:3037] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:15:19.036696 10268 raft_consensus.cc:2454] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d02e77b2be6476e891555c8cd2f5d17 in term 1.
I20260806 01:15:19.037153 10588 leader_election.cc:302] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d02e77b2be6476e891555c8cd2f5d17, 895dad9fd46b417e951d7f20caac9a0c; no voters:
I20260806 01:15:19.037472 10800 raft_consensus.cc:2781] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:15:19.037500 10474 raft_consensus.cc:2454] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d02e77b2be6476e891555c8cd2f5d17 in term 1.
I20260806 01:15:19.038300 10800 raft_consensus.cc:686] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [term 1 LEADER]: Becoming Leader. State: Replica: 5d02e77b2be6476e891555c8cd2f5d17, State: Running, Role: LEADER
I20260806 01:15:19.038946 10800 consensus_queue.cc:237] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895dad9fd46b417e951d7f20caac9a0c" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 40087 } } peers { permanent_uuid: "5d02e77b2be6476e891555c8cd2f5d17" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 45755 } } peers { permanent_uuid: "cce38cc01e854b5582c4c96da682a7ca" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 34877 } }
I20260806 01:15:19.044909 10033 catalog_manager.cc:5495] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d02e77b2be6476e891555c8cd2f5d17 (127.25.236.131). New cstate: current_term: 1 leader_uuid: "5d02e77b2be6476e891555c8cd2f5d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895dad9fd46b417e951d7f20caac9a0c" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 40087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d02e77b2be6476e891555c8cd2f5d17" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 45755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cce38cc01e854b5582c4c96da682a7ca" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 34877 } health_report { overall_health: UNKNOWN } } }
I20260806 01:15:19.072494 10789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:19.066540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51640 (local address 127.25.236.131:45755)
0806 01:15:19.066762 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:19.066767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:19.066946 (+ 179us) server_negotiation.cc:407] Connection header received
0806 01:15:19.067124 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:19.067129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:19.067229 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:19.067380 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:15:19.067765 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.068725 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.070198 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.070681 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.070792 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:19.071363 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:15:19.071908 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:15:19.072023 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:19.072207 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 01:15:19.072281 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:15:19.082510 10268 raft_consensus.cc:1261] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c [term 1 FOLLOWER]: Refusing update from remote peer 5d02e77b2be6476e891555c8cd2f5d17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:15:19.082799 10474 raft_consensus.cc:1261] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca [term 1 FOLLOWER]: Refusing update from remote peer 5d02e77b2be6476e891555c8cd2f5d17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:15:19.083803 10800 consensus_queue.cc:1035] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "895dad9fd46b417e951d7f20caac9a0c" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 40087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:15:19.084455 10805 consensus_queue.cc:1035] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cce38cc01e854b5582c4c96da682a7ca" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:15:19.095711 10762 tablet.cc:2369] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:15:19.115310 10033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:15:19.115613 10033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:15:19.120558 10033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bd2bdf4bd2149abba9274729a075594: Sending DeleteTablet for 3 replicas of tablet 099e92acc3b94b2b913ba35ac3fe5aae
I20260806 01:15:19.121865 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet 099e92acc3b94b2b913ba35ac3fe5aae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:15:19 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:51630
I20260806 01:15:19.121857 10234 tablet_service.cc:1514] Processing DeleteTablet for tablet 099e92acc3b94b2b913ba35ac3fe5aae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:15:19 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:32778
I20260806 01:15:19.122740 10446 tablet_service.cc:1514] Processing DeleteTablet for tablet 099e92acc3b94b2b913ba35ac3fe5aae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:15:19 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:35260
I20260806 01:15:19.123756 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 10132
I20260806 01:15:19.123777 10825 tablet_replica.cc:330] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17: stopping tablet replica
I20260806 01:15:19.124226 10825 raft_consensus.cc:2229] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:15:19.124925 10825 raft_consensus.cc:2258] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:15:19.125649 10824 tablet_replica.cc:330] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c: stopping tablet replica
I20260806 01:15:19.126192 10824 raft_consensus.cc:2229] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:15:19.127043 10824 raft_consensus.cc:2258] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:15:19.127969 10825 ts_tablet_manager.cc:1905] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:15:19.134652 10824 ts_tablet_manager.cc:1905] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:15:19.136181 10828 tablet_replica.cc:330] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca: stopping tablet replica
I20260806 01:15:19.136709 10828 raft_consensus.cc:2229] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:15:19.137620 10828 raft_consensus.cc:2258] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:15:19.138286 10825 ts_tablet_manager.cc:1918] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:15:19.138545 10825 log.cc:1196] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/099e92acc3b94b2b913ba35ac3fe5aae
I20260806 01:15:19.139256 10825 ts_tablet_manager.cc:1939] T 099e92acc3b94b2b913ba35ac3fe5aae P 5d02e77b2be6476e891555c8cd2f5d17: Deleting consensus metadata
I20260806 01:15:19.141376 10009 catalog_manager.cc:4841] TS 5d02e77b2be6476e891555c8cd2f5d17 (127.25.236.131:45755): tablet 099e92acc3b94b2b913ba35ac3fe5aae (table test-table [id=bd85b34dea05457399911404b956bf02]) successfully deleted
I20260806 01:15:19.142935 10824 ts_tablet_manager.cc:1918] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:15:19.143390 10824 log.cc:1196] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/099e92acc3b94b2b913ba35ac3fe5aae
I20260806 01:15:19.144225 10824 ts_tablet_manager.cc:1939] T 099e92acc3b94b2b913ba35ac3fe5aae P 895dad9fd46b417e951d7f20caac9a0c: Deleting consensus metadata
I20260806 01:15:19.145087 10828 ts_tablet_manager.cc:1905] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:15:19.146670 10013 catalog_manager.cc:4841] TS 895dad9fd46b417e951d7f20caac9a0c (127.25.236.129:40087): tablet 099e92acc3b94b2b913ba35ac3fe5aae (table test-table [id=bd85b34dea05457399911404b956bf02]) successfully deleted
Aug 06 01:15:19 dist-test-slave-7nm7 krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:15:19.153441 10828 ts_tablet_manager.cc:1918] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:15:19.153644 10828 log.cc:1196] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/099e92acc3b94b2b913ba35ac3fe5aae
I20260806 01:15:19.154309 10828 ts_tablet_manager.cc:1939] T 099e92acc3b94b2b913ba35ac3fe5aae P cce38cc01e854b5582c4c96da682a7ca: Deleting consensus metadata
I20260806 01:15:19.156558 10010 catalog_manager.cc:4841] TS cce38cc01e854b5582c4c96da682a7ca (127.25.236.130:34877): tablet 099e92acc3b94b2b913ba35ac3fe5aae (table test-table [id=bd85b34dea05457399911404b956bf02]) successfully deleted
I20260806 01:15:19.157922 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:19.126833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32786 (local address 127.25.236.129:40087)
0806 01:15:19.127012 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:19.127026 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:19.133066 (+ 6040us) server_negotiation.cc:407] Connection header received
0806 01:15:19.134112 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:19.134118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:19.134205 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:19.134306 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:19.138785 (+ 4479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.140016 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.142618 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.143309 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.143470 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:19.149445 (+ 5975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:19.149462 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:19.149480 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:15:19.149536 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:19.153923 (+ 4387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:19.155080 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:19.155101 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:19.155105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:19.155282 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:19.155746 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:19.155751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:19.155754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:19.156249 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:19.156581 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:19.157296 (+ 715us) server_negotiation.cc:299] Negotiation successful
0806 01:15:19.157534 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:15:19.158682 10329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:15:19.293723 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 10132
I20260806 01:15:19.326668 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 10346
Aug 06 01:15:19 dist-test-slave-7nm7 krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:15:19.342700 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:19.327822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35274 (local address 127.25.236.130:34877)
0806 01:15:19.328001 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:19.328007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:19.328029 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:15:19.328194 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:19.328198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:19.328266 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:19.328377 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:19.329606 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.330450 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.331688 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.332085 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.332225 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:19.336461 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:19.336475 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:19.336492 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:15:19.336525 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:19.339536 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:19.340324 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:19.340340 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:19.340342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:19.340465 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:19.340864 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:19.340871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:19.340875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:19.341281 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:19.341703 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:19.342025 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 01:15:19.342299 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 01:15:19.343844 10544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:15:19.447932 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 10346
I20260806 01:15:19.475651 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 10561
Aug 06 01:15:19 dist-test-slave-7nm7 krb5kdc[9967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:15:19.494907 10789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:15:19.476592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51650 (local address 127.25.236.131:45755)
0806 01:15:19.476812 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 01:15:19.476818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:15:19.476841 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:15:19.477034 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:15:19.477040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:15:19.477128 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:15:19.477253 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:15:19.479426 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.480776 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.482122 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:15:19.482963 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:15:19.483128 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:15:19.487826 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:15:19.487839 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:15:19.487856 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:15:19.487895 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:15:19.491473 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:19.492513 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:19.492521 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:19.492523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:19.492682 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:15:19.493113 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:15:19.493119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:15:19.493121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:15:19.493514 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:15:19.493764 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 01:15:19.494168 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 01:15:19.494499 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 01:15:19.495718 10755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:15:19.623262 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 10561
I20260806 01:15:19.655283 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 9984
I20260806 01:15:19.656601 10086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:15:19.777540 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 9984
2026-08-06T01:15:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5177 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:15:19 dist-test-slave-7nm7 krb5kdc[10838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:15:19 dist-test-slave-7nm7 krb5kdc[10838](info): set up 2 sockets
Aug 06 01:15:19 dist-test-slave-7nm7 krb5kdc[10838](info): commencing operation
krb5kdc: starting...
W20260806 01:15:21.201715 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:15:21 dist-test-slave-7nm7 krb5kdc[10838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:15:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:15:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/rangeradmin_127.25.236.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/rangeradmin_127.25.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/rangerlookup_127.25.236.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/rangerlookup_127.25.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/HTTP_127.25.236.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/HTTP_127.25.236.148@KRBTEST.COM.keytab.
I20260806 01:15:21.684435 26546 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskVUGvte/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/postgres -l logfile start
2026-08-06 01:15:24.514 UTC [10884] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 01:15:24.514 UTC [10884] LOG: listening on IPv4 address "127.25.236.148", port 57307
2026-08-06 01:15:24.518 UTC [10884] LOG: listening on Unix socket "/tmp/.s.PGSQL.57307"
2026-08-06 01:15:24.542 UTC [10889] LOG: database system was shut down at 2026-08-06 01:15:22 UTC
2026-08-06 01:15:24.547 UTC [10884] LOG: database system is ready to accept connections
I20260806 01:15:25.790750 26546 mini_postgres.cc:96] Postgres bound to 57307
2026-08-06 01:15:25.821 UTC [10901] FATAL: database "slave" does not exist
127.25.236.148:57307 - accepting connections
I20260806 01:15:25.822515 26546 mini_ranger.cc:161] Starting Ranger...
I20260806 01:15:25.862684 26546 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 01:15:26.347554 26546 mini_ranger.cc:87] Created ranger Postgres database
I20260806 01:15:26.347720 26546 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-admin
2026-08-06 01:15:26,799 [I] DB FLAVOR :POSTGRES
2026-08-06 01:15:26,800 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 01:15:26,800 [I] Checking connection..
2026-08-06 01:15:26,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 01:15:27,114 [I] Checking connection passed.
2026-08-06 01:15:27,114 [I] --------- Verifying version history table ---------
2026-08-06 01:15:27,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:15:27,394 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 01:15:27,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:15:27,658 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 01:15:27,659 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 01:15:27,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 01:15:27.924 UTC [11008] WARNING: there is no transaction in progress
2026-08-06 01:15:27,934 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 01:15:27,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:15:28,213 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 01:15:28,213 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 01:15:28,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 01:15:28,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 01:15:28,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7nm7.c.gcp-upstream.internal','N') ;"
2026-08-06 01:15:29,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 01:15:29,316 [I] Table x_portal_user does not exist in database ranger
2026-08-06 01:15:29,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 01:15:29,601 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 01:15:29,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 01:15:29,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 01:15:30,181 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 01:15:30,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 01:15:31.385 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.403 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.421 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.437 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.454 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.475 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.534 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.541 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.558 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.570 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.582 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.597 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.607 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.614 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.620 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:31.627 UTC [11169] WARNING: there is no transaction in progress
2026-08-06 01:15:32,395 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 01:15:32,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 01:15:32,676 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 01:15:32,676 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 01:15:32,965 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 01:15:32,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 01:15:33,254 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 01:15:33,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 01:15:33,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 01:15:33,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7nm7.c.gcp-upstream.internal';"
2026-08-06 01:15:34,061 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 01:15:34,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 01:15:34,339 [I] DB_PATCHES have already been applied
I20260806 01:15:34.346009 26546 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-admin:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskVUGvte/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 01:15:34.346120 26546 mini_ranger.cc:193] Using host: 127.25.236.148
I20260806 01:15:34.372035 26546 mini_ranger.cc:227] Ranger admin URL: http://127.25.236.148:53377
Aug 06, 2026 1:15:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:15:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:15:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 1:15:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 1:15:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 1:15:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 1:15:35 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 01:15:35.836799 26546 mini_ranger.cc:160] Time spent starting Ranger: real 10.014s user 0.007s sys 0.076s
Aug 06, 2026 1:15:58 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 1:15:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 06, 2026 1:15:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 1:15:58 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 1:15:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 1:15:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 1:15:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 1:15:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 1:15:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 1:15:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 1:15:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 1:15:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 1:15:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 1:15:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 1:15:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 1:15:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 1:15:58 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 1:15:59 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 01:16:00.319832 26546 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/rangerkms_127.25.236.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/rangerkms_127.25.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.236.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.236.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.236.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/HTTP_127.25.236.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.236.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/HTTP_127.25.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 01:16:00 dist-test-slave-7nm7 krb5kdc[10838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978960, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 01:16:00.585225 26546 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 01:16:00.625293 26546 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 01:16:01.681574 26546 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 01:16:01.681763 26546 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-kms
I20260806 01:16:01.681896 26546 mini_ranger_kms.cc:227] Using postgres at 127.25.236.148:57307
2026-08-06 01:16:02,242 [I] DB FLAVOR :POSTGRES
2026-08-06 01:16:02,243 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 01:16:02,243 [I] Checking connection
2026-08-06 01:16:02,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 01:16:02,533 [I] connection success
2026-08-06 01:16:02,533 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 01:16:02,534 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 01:16:02,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 01:16:02,823 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 01:16:02,824 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 01:16:02,824 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 01:16:02,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:57307/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 01:16:03,150 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 01:16:03.780742 26546 mini_ranger_kms.cc:310] Created kms service
I20260806 01:16:04.009148 26546 mini_ranger_kms.cc:326] Created kudu user
I20260806 01:16:04.061911 26546 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 01:16:04 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978960, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:16:04.593873 26546 mini_ranger_kms.cc:384] Added ranger policy
I20260806 01:16:04.595291 26546 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-kms:/tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 01:16:04.595409 26546 mini_ranger_kms.cc:240] Using host: 127.25.236.148
I20260806 01:16:04.630759 26546 mini_ranger_kms.cc:276] Ranger KMS PID: 11461
I20260806 01:16:04.630983 26546 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.236.148:50815
Aug 06, 2026 1:16:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:16:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:16:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 1:16:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 1:16:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 1:16:06 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 01:16:07 dist-test-slave-7nm7 krb5kdc[10838](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785978967, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.236.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 01:16:07 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978967, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.236.148@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:16:07.583647 26546 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.998s user 0.169s sys 0.446s
I20260806 01:16:07.584095 26546 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 01:16:07.584323 26546 mini_ranger_kms.cc:401] 127.25.236.148:50815/kms/v1/keys
Aug 06 01:16:07 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978967, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.236.148@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
Aug 06 01:16:10 dist-test-slave-7nm7 krb5kdc[10838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:16:10.529575 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:32969
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:32969
--ranger_config_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.236.148:50815/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:16:10.964624 11546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:16:10.964963 11546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:16:10.965173 11546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:16:10.973147 11546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:16:10.973330 11546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:16:10.973457 11546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:16:10.973578 11546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:16:10.973675 11546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:16:10.984429 11546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.236.148:50815/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:32969
--ranger_config_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:32969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:16:10.986107 11546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:16:10.987586 11546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:16:11.000169 11559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:16:11.000245 11556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:16:11.000270 11555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:16:11.000466 11546 server_base.cc:1034] running on GCE node
I20260806 01:16:11.001507 11546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:16:11.002923 11546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:16:11.004137 11546 hybrid_clock.cc:648] HybridClock initialized: now 1785978971004082 us; error 38 us; skew 500 ppm
I20260806 01:16:11.004459 11546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:16:11 dist-test-slave-7nm7 krb5kdc[10838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978971, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:16:11.007652 11546 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:16:11.009286 11546 webserver.cc:460] Webserver started at http://127.25.236.190:43363/ using document root <none> and password file <none>
I20260806 01:16:11.009946 11546 fs_manager.cc:359] Metadata directory not provided
I20260806 01:16:11.010098 11546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:16:11.010496 11546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:16:11 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978971, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:16:11.062646 11546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "45dfa5c2c7ed4065a3f33212875f1c0a"
format_stamp: "Formatted at 2026-08-06 01:16:11 on dist-test-slave-7nm7"
server_key: "8a5fe045294669b714194c3c9b4e2686"
server_key_iv: "49b114f4768bb748604ddf8ae9dcdd74"
server_key_version: "kuduclusterkey@0"
I20260806 01:16:11.063894 11546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "45dfa5c2c7ed4065a3f33212875f1c0a"
format_stamp: "Formatted at 2026-08-06 01:16:11 on dist-test-slave-7nm7"
server_key: "8a5fe045294669b714194c3c9b4e2686"
server_key_iv: "49b114f4768bb748604ddf8ae9dcdd74"
server_key_version: "kuduclusterkey@0"
I20260806 01:16:11.071910 11546 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 01:16:11.094949 11570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:16:11.096891 11546 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 01:16:11.097122 11546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "45dfa5c2c7ed4065a3f33212875f1c0a"
format_stamp: "Formatted at 2026-08-06 01:16:11 on dist-test-slave-7nm7"
server_key: "8a5fe045294669b714194c3c9b4e2686"
server_key_iv: "49b114f4768bb748604ddf8ae9dcdd74"
server_key_version: "kuduclusterkey@0"
I20260806 01:16:11.097414 11546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:16:11.116534 11546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:16:11.122030 11546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:16:11.122447 11546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:16:11.145871 11649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:32969 every 8 connection(s)
I20260806 01:16:11.145879 11546 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:32969
I20260806 01:16:11.147892 11546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:16:11.151175 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 11546
I20260806 01:16:11.151407 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:16:11.152246 26546 external_mini_cluster.cc:1428] Setting key a075ca6f036c439d3e336616b1640cac
I20260806 01:16:11.152505 11650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:16:11.169664 11650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a: Bootstrap starting.
I20260806 01:16:11.173205 11650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a: Neither blocks nor log segments found. Creating new log.
I20260806 01:16:11.174571 11650 log.cc:824] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:16:11.178097 11650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a: No bootstrap required, opened a new log
Aug 06 01:16:11 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:16:11.188696 11650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45dfa5c2c7ed4065a3f33212875f1c0a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 32969 } }
I20260806 01:16:11.189129 11650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:16:11.189270 11650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45dfa5c2c7ed4065a3f33212875f1c0a, State: Initialized, Role: FOLLOWER
I20260806 01:16:11.190058 11650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45dfa5c2c7ed4065a3f33212875f1c0a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 32969 } }
I20260806 01:16:11.190403 11650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:16:11.190567 11650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:16:11.190806 11650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:16:11.193918 11650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45dfa5c2c7ed4065a3f33212875f1c0a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 32969 } }
I20260806 01:16:11.194590 11650 leader_election.cc:302] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45dfa5c2c7ed4065a3f33212875f1c0a; no voters:
I20260806 01:16:11.194841 11657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:11.154411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34402 (local address 127.25.236.190:32969)
0806 01:16:11.167089 (+ 12678us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:11.167113 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:11.168727 (+ 1614us) server_negotiation.cc:407] Connection header received
0806 01:16:11.171554 (+ 2827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:11.171582 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:11.172097 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:11.172552 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:16:11.174316 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:11.176452 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:11.178352 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:11.179708 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:11.179917 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:11.185602 (+ 5685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:16:11.185613 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:16:11.185635 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:16:11.185688 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:16:11.189718 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:11.190617 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:11.190626 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:11.190634 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:11.190823 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:11.191349 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:11.191354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:11.191358 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:11.192126 (+ 768us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:16:11.192419 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:11.192922 (+ 503us) server_negotiation.cc:299] Negotiation successful
0806 01:16:11.193345 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12233,"thread_start_us":11911,"threads_started":1}
I20260806 01:16:11.198313 11650 leader_election.cc:288] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:16:11.198390 11660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:16:11.200338 11650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:16:11.210553 11660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [term 1 LEADER]: Becoming Leader. State: Replica: 45dfa5c2c7ed4065a3f33212875f1c0a, State: Running, Role: LEADER
I20260806 01:16:11.211320 11660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45dfa5c2c7ed4065a3f33212875f1c0a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 32969 } }
I20260806 01:16:11.215773 11663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45dfa5c2c7ed4065a3f33212875f1c0a. Latest consensus state: current_term: 1 leader_uuid: "45dfa5c2c7ed4065a3f33212875f1c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45dfa5c2c7ed4065a3f33212875f1c0a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 32969 } } }
I20260806 01:16:11.216199 11663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [sys.catalog]: This master's current role is: LEADER
I20260806 01:16:11.216811 11661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45dfa5c2c7ed4065a3f33212875f1c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45dfa5c2c7ed4065a3f33212875f1c0a" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 32969 } } }
I20260806 01:16:11.217145 11661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a [sys.catalog]: This master's current role is: LEADER
I20260806 01:16:11.230978 11685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:16:11.231153 11650 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 01:16:11.247965 11685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:16:11.270033 11685 catalog_manager.cc:1347] Generated new cluster ID: 189e5d376a694b44a8bfe550f639ba98
I20260806 01:16:11.270167 11685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:16:11.315574 11685 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:16:11.317178 11685 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:16:11.326162 11685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a: Generated new TSK 0
I20260806 01:16:11.327669 11685 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:16:12.170042 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.236.148:50815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:32969
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:16:12 dist-test-slave-7nm7 krb5kdc[10838](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785978972, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 01:16:12.694067 11725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:16:12.694545 11725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:16:12.694733 11725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:16:12.707643 11725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:16:12.707942 11725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:16:12.708194 11725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:16:12.726091 11725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.236.148:50815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:32969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:16:12.728127 11725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:16:12.729904 11725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:16:12.748493 11743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:16:12.750022 11744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:16:12.758538 11725 server_base.cc:1034] running on GCE node
W20260806 01:16:12.759629 11748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:16:12.760653 11725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:16:12.766660 11725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:16:12.768005 11725 hybrid_clock.cc:648] HybridClock initialized: now 1785978972767951 us; error 49 us; skew 500 ppm
I20260806 01:16:12.768494 11725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:16:12 dist-test-slave-7nm7 krb5kdc[10838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978972, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:16:12.773787 11725 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:16:12.783391 11725 webserver.cc:460] Webserver started at http://127.25.236.129:38875/ using document root <none> and password file <none>
I20260806 01:16:12.785013 11725 fs_manager.cc:359] Metadata directory not provided
I20260806 01:16:12.785284 11725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:16:12.785853 11725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:16:12 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978972, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:16:12.817896 11725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb"
format_stamp: "Formatted at 2026-08-06 01:16:12 on dist-test-slave-7nm7"
server_key: "d776215ba0b3b63defc0c1c21c8da8da"
server_key_iv: "7e196330f313e4569d4932d476d23d06"
server_key_version: "kuduclusterkey@0"
I20260806 01:16:12.819063 11725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb"
format_stamp: "Formatted at 2026-08-06 01:16:12 on dist-test-slave-7nm7"
server_key: "d776215ba0b3b63defc0c1c21c8da8da"
server_key_iv: "7e196330f313e4569d4932d476d23d06"
server_key_version: "kuduclusterkey@0"
I20260806 01:16:12.828374 11725 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.001s
I20260806 01:16:12.852011 11759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:16:12.860713 11725 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.005s sys 0.000s
I20260806 01:16:12.861090 11725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb"
format_stamp: "Formatted at 2026-08-06 01:16:12 on dist-test-slave-7nm7"
server_key: "d776215ba0b3b63defc0c1c21c8da8da"
server_key_iv: "7e196330f313e4569d4932d476d23d06"
server_key_version: "kuduclusterkey@0"
I20260806 01:16:12.861785 11725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:16:12.950825 11725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:16:12.958340 11725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:16:12.958850 11725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:16:12.966507 11725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:16:12.978811 11725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:16:12.979048 11725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 01:16:12.979295 11725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:16:12.979450 11725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:16:13 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978972, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
Aug 06 01:16:13 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978972, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:16:13.717703 11935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:38249 every 8 connection(s)
I20260806 01:16:13.718120 11725 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:38249
I20260806 01:16:13.720147 11725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:16:13.725955 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 11725
I20260806 01:16:13.726197 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:16:13.727111 26546 external_mini_cluster.cc:1428] Setting key fd5c0b718a999c17c5eaebe836a782f0
Aug 06 01:16:13 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978972, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:16:13.791388 11939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:13.732695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:35729 (local address 127.25.236.190:32969)
0806 01:16:13.756962 (+ 24267us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:13.756969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:13.762297 (+ 5328us) server_negotiation.cc:407] Connection header received
0806 01:16:13.762388 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:13.762394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:13.762490 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:13.762607 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:16:13.766375 (+ 3768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:13.767574 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:13.770319 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:13.771052 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:13.771172 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:13.782262 (+ 11090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:16:13.782272 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:16:13.782278 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:16:13.782332 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:16:13.786665 (+ 4333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:13.787844 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:13.787852 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:13.787856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:13.788043 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:13.788691 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:13.788697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:13.788700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:13.789326 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:16:13.789466 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:13.790596 (+ 1130us) server_negotiation.cc:299] Negotiation successful
0806 01:16:13.790866 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24095,"thread_start_us":24308,"threads_started":1}
I20260806 01:16:13.794210 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:13.732159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:32969 (local address 127.25.236.129:35729)
0806 01:16:13.758684 (+ 26525us) negotiation.cc:107] Waiting for socket to connect
0806 01:16:13.758758 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 01:16:13.761449 (+ 2691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:16:13.764477 (+ 3028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:16:13.764491 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:16:13.765214 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:16:13.766091 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:13.766118 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:13.767838 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:13.767844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:13.770065 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:13.770084 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:13.771590 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:13.771598 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:13.771914 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:13.778334 (+ 6420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:16:13.778380 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:16:13.781834 (+ 3454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:16:13.786912 (+ 5078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:16:13.786923 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:16:13.786944 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:16:13.787661 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:16:13.788225 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:16:13.788230 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:16:13.788234 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:16:13.788530 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:16:13.789592 (+ 1062us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:16:13.789601 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:16:13.790084 (+ 483us) client_negotiation.cc:769] Sending connection context
0806 01:16:13.792041 (+ 1957us) client_negotiation.cc:240] Negotiation successful
0806 01:16:13.792656 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26139,"thread_start_us":29643,"threads_started":1}
I20260806 01:16:13.796633 11936 heartbeater.cc:347] Connected to a master server at 127.25.236.190:32969
I20260806 01:16:13.797010 11936 heartbeater.cc:464] Registering TS with master...
I20260806 01:16:13.798004 11936 heartbeater.cc:511] Master 127.25.236.190:32969 requested a full tablet report, sending...
I20260806 01:16:13.801013 11598 ts_manager.cc:194] Registered new tserver with Master: f6c3fc311c2c47c8b5031b453cb1d7cb (127.25.236.129:38249)
I20260806 01:16:13.803629 11598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:35729
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
W20260806 01:16:13.905515 26546 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.25.236.130: real 0.175s user 0.001s sys 0.025s
I20260806 01:16:13.906466 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.236.148:50815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:32969
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:16:14.394706 11943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:16:14.394982 11943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:16:14.395179 11943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:16:14.403261 11943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:16:14.403396 11943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:16:14.403601 11943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:16:14.414888 11943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.236.148:50815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:32969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:16:14.416252 11943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:16:14.417582 11943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:16:14.434592 11963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:16:14.434517 11964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:16:14.434475 11967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:16:14.435770 11943 server_base.cc:1034] running on GCE node
I20260806 01:16:14.436439 11943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:16:14.437463 11943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:16:14.438725 11943 hybrid_clock.cc:648] HybridClock initialized: now 1785978974438668 us; error 34 us; skew 500 ppm
I20260806 01:16:14.439182 11943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:16:14 dist-test-slave-7nm7 krb5kdc[10838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978974, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:16:14.443327 11943 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:16:14.445341 11943 webserver.cc:460] Webserver started at http://127.25.236.130:32899/ using document root <none> and password file <none>
I20260806 01:16:14.446220 11943 fs_manager.cc:359] Metadata directory not provided
I20260806 01:16:14.446419 11943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:16:14.446785 11943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:16:14 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978974, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:16:14.465756 11943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "e4f03b223b4a41c0a550ca13e695468f"
format_stamp: "Formatted at 2026-08-06 01:16:14 on dist-test-slave-7nm7"
server_key: "a62c9250214eb6b90ece4979d83ae0df"
server_key_iv: "bb60d5f7e1ad9640c10715692cd6d7a2"
server_key_version: "kuduclusterkey@0"
I20260806 01:16:14.467108 11943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "e4f03b223b4a41c0a550ca13e695468f"
format_stamp: "Formatted at 2026-08-06 01:16:14 on dist-test-slave-7nm7"
server_key: "a62c9250214eb6b90ece4979d83ae0df"
server_key_iv: "bb60d5f7e1ad9640c10715692cd6d7a2"
server_key_version: "kuduclusterkey@0"
I20260806 01:16:14.475037 11943 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260806 01:16:14.491042 11976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:16:14.492815 11943 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:16:14.493048 11943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "e4f03b223b4a41c0a550ca13e695468f"
format_stamp: "Formatted at 2026-08-06 01:16:14 on dist-test-slave-7nm7"
server_key: "a62c9250214eb6b90ece4979d83ae0df"
server_key_iv: "bb60d5f7e1ad9640c10715692cd6d7a2"
server_key_version: "kuduclusterkey@0"
I20260806 01:16:14.493287 11943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:16:14.507620 11943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:16:14.512980 11943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:16:14.513370 11943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:16:14.514570 11943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:16:14.516243 11943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:16:14.516327 11943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:16:14.516525 11943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:16:14.516585 11943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:16:14.558540 11943 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:35781
I20260806 01:16:14.558573 12140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:35781 every 8 connection(s)
I20260806 01:16:14.559953 11943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:16:14.561884 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 11943
I20260806 01:16:14.562100 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:16:14.562754 26546 external_mini_cluster.cc:1428] Setting key 8c06b87a0b649c9324e46353f210caf5
Aug 06 01:16:14 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978974, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:16:14.587769 12146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:14.563936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:45025 (local address 127.25.236.190:32969)
0806 01:16:14.564491 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:14.564496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:14.565825 (+ 1329us) server_negotiation.cc:407] Connection header received
0806 01:16:14.568027 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:14.568033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:14.568287 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:14.568518 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:16:14.571284 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:14.572071 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:14.573748 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:14.574159 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:14.574318 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:14.578723 (+ 4405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:16:14.578734 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:16:14.578742 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:16:14.578800 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:16:14.583184 (+ 4384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:14.584216 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:14.584225 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:14.584232 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:14.584429 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:14.585102 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:14.585110 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:14.585116 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:14.585576 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:16:14.585749 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:14.586983 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0806 01:16:14.587262 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":437,"threads_started":1}
I20260806 01:16:14.588444 12150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:14.564492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:32969 (local address 127.25.236.130:45025)
0806 01:16:14.565661 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0806 01:16:14.565721 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 01:16:14.567745 (+ 2024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:16:14.568983 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:16:14.568997 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:16:14.569766 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:16:14.571005 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:14.571031 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:14.572262 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:14.572266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:14.573594 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:14.573606 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:14.574416 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:14.574423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:14.574669 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:14.575492 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:16:14.575517 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:16:14.578408 (+ 2891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:16:14.583419 (+ 5011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:16:14.583428 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:16:14.583444 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:16:14.584057 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:16:14.584587 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:16:14.584592 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:16:14.584597 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:16:14.584953 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:16:14.585760 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:16:14.585766 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:16:14.586497 (+ 731us) client_negotiation.cc:769] Sending connection context
0806 01:16:14.586821 (+ 324us) client_negotiation.cc:240] Negotiation successful
0806 01:16:14.587266 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":407,"threads_started":1}
I20260806 01:16:14.590147 12143 heartbeater.cc:347] Connected to a master server at 127.25.236.190:32969
I20260806 01:16:14.590561 12143 heartbeater.cc:464] Registering TS with master...
I20260806 01:16:14.591665 12143 heartbeater.cc:511] Master 127.25.236.190:32969 requested a full tablet report, sending...
I20260806 01:16:14.593606 11598 ts_manager.cc:194] Registered new tserver with Master: e4f03b223b4a41c0a550ca13e695468f (127.25.236.130:35781)
I20260806 01:16:14.594686 11598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:45025
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:16:14.642707 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.236.148:50815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:32969
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:16:14.807874 11936 heartbeater.cc:503] Master 127.25.236.190:32969 was elected leader, sending a full tablet report...
W20260806 01:16:15.039742 12153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:16:15.040019 12153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:16:15.040208 12153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:16:15.047572 12153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:16:15.047679 12153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:16:15.047892 12153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:16:15.058297 12153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.236.148:50815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:32969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:16:15.059712 12153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:16:15.061002 12153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:16:15.075881 12163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:16:15.075961 12164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:16:15.075917 12166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:16:15.077039 12153 server_base.cc:1034] running on GCE node
I20260806 01:16:15.077800 12153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:16:15.079161 12153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:16:15.080416 12153 hybrid_clock.cc:648] HybridClock initialized: now 1785978975080334 us; error 68 us; skew 500 ppm
I20260806 01:16:15.080875 12153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:16:15 dist-test-slave-7nm7 krb5kdc[10838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978975, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:16:15.085675 12153 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:16:15.087611 12153 webserver.cc:460] Webserver started at http://127.25.236.131:34957/ using document root <none> and password file <none>
I20260806 01:16:15.088444 12153 fs_manager.cc:359] Metadata directory not provided
I20260806 01:16:15.088586 12153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:16:15.088932 12153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:16:15 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978975, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:16:15.104475 12153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "a60ddb6b9b38466dbad7338a7d34b963"
format_stamp: "Formatted at 2026-08-06 01:16:15 on dist-test-slave-7nm7"
server_key: "6256501c84bcacfb3a2e781a6e55a669"
server_key_iv: "d71d1764f9100d30cee303ed046001d0"
server_key_version: "kuduclusterkey@0"
I20260806 01:16:15.105657 12153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "a60ddb6b9b38466dbad7338a7d34b963"
format_stamp: "Formatted at 2026-08-06 01:16:15 on dist-test-slave-7nm7"
server_key: "6256501c84bcacfb3a2e781a6e55a669"
server_key_iv: "d71d1764f9100d30cee303ed046001d0"
server_key_version: "kuduclusterkey@0"
I20260806 01:16:15.112871 12153 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.003s
I20260806 01:16:15.127746 12176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:16:15.129305 12153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:16:15.129493 12153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "a60ddb6b9b38466dbad7338a7d34b963"
format_stamp: "Formatted at 2026-08-06 01:16:15 on dist-test-slave-7nm7"
server_key: "6256501c84bcacfb3a2e781a6e55a669"
server_key_iv: "d71d1764f9100d30cee303ed046001d0"
server_key_version: "kuduclusterkey@0"
I20260806 01:16:15.129712 12153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:16:15.143401 12153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:16:15.147686 12153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:16:15.148000 12153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:16:15.149065 12153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:16:15.150585 12153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:16:15.150658 12153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:16:15.150831 12153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:16:15.150883 12153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:16:15.190817 12153 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:39045
I20260806 01:16:15.190867 12352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:39045 every 8 connection(s)
I20260806 01:16:15.192114 12153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:16:15.198896 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 12153
I20260806 01:16:15.199115 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:16:15.199750 26546 external_mini_cluster.cc:1428] Setting key 487c7a36ae9686d110045230447f8c43
Aug 06 01:16:15 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978975, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:16:15.215690 12360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.194768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:45529 (local address 127.25.236.190:32969)
0806 01:16:15.195426 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:15.195430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:15.196295 (+ 865us) server_negotiation.cc:407] Connection header received
0806 01:16:15.198641 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:15.198646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:15.198718 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:15.198817 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:16:15.200964 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.202105 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.204217 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.204689 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.204857 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.208814 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:16:15.208819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:16:15.208822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:16:15.208936 (+ 114us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:16:15.212429 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:15.213203 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:15.213216 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:15.213218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:15.213336 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:15.213850 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:15.213856 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:15.213861 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:15.214155 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:16:15.214347 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:15.215099 (+ 752us) server_negotiation.cc:299] Negotiation successful
0806 01:16:15.215319 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":347,"threads_started":1}
I20260806 01:16:15.216876 12361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.195385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:32969 (local address 127.25.236.131:45529)
0806 01:16:15.196120 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0806 01:16:15.196168 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:16:15.198291 (+ 2123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:16:15.199142 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:16:15.199155 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:16:15.199796 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:16:15.200762 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:15.200784 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.202336 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.202343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:15.204037 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:15.204060 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.204955 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.204964 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:15.205227 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.205995 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:16:15.206021 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:16:15.208580 (+ 2559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:16:15.212601 (+ 4021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:16:15.212611 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:16:15.212634 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:16:15.213067 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:16:15.213451 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:16:15.213456 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:16:15.213460 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:16:15.213721 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:16:15.214328 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:16:15.214336 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:16:15.214680 (+ 344us) client_negotiation.cc:769] Sending connection context
0806 01:16:15.214947 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 01:16:15.215471 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":308,"threads_started":1}
I20260806 01:16:15.218559 12357 heartbeater.cc:347] Connected to a master server at 127.25.236.190:32969
I20260806 01:16:15.218861 12357 heartbeater.cc:464] Registering TS with master...
I20260806 01:16:15.219673 12357 heartbeater.cc:511] Master 127.25.236.190:32969 requested a full tablet report, sending...
I20260806 01:16:15.221204 11598 ts_manager.cc:194] Registered new tserver with Master: a60ddb6b9b38466dbad7338a7d34b963 (127.25.236.131:39045)
I20260806 01:16:15.222080 11598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:45529
I20260806 01:16:15.229177 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:16:15.251638 12360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.234904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48918 (local address 127.25.236.190:32969)
0806 01:16:15.235282 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:15.235290 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:15.235637 (+ 347us) server_negotiation.cc:407] Connection header received
0806 01:16:15.235855 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:15.235861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:15.235947 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:15.236057 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:16:15.237637 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.238481 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.239833 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.240586 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.240876 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.244093 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:16:15.244100 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:16:15.244103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:16:15.244148 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:16:15.247990 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:15.248802 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:15.248809 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:15.248812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:15.249009 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:15.249604 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:15.249612 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:15.249615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:15.250034 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:16:15.250418 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:15.250813 (+ 395us) server_negotiation.cc:299] Negotiation successful
0806 01:16:15.251067 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:16:15.257073 11598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:16:15.267108 11598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:16:15.313669 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.286841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51872 (local address 127.25.236.131:39045)
0806 01:16:15.304756 (+ 17915us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:15.304764 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:15.304795 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:16:15.304888 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:15.304897 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:15.305117 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:15.305339 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:16:15.306532 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.308059 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.310987 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.312314 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.312532 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.312742 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:15.313105 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 01:16:15.313294 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17682,"thread_start_us":17534,"threads_started":1}
I20260806 01:16:15.313732 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.293509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:39045 (local address 127.0.0.1:51872)
0806 01:16:15.303960 (+ 10451us) negotiation.cc:107] Waiting for socket to connect
0806 01:16:15.303982 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:16:15.304102 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:16:15.305645 (+ 1543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:16:15.305649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:16:15.305669 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:16:15.306262 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:15.306278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.308230 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.308237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:15.310767 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:15.310783 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.312521 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.312527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:15.312734 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.312799 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 01:16:15.312946 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 01:16:15.313250 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10322,"mutex_wait_us":91,"thread_start_us":7506,"threads_started":1}
I20260806 01:16:15.314610 12382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.285734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:38249 (local address 127.0.0.1:45544)
0806 01:16:15.301378 (+ 15644us) negotiation.cc:107] Waiting for socket to connect
0806 01:16:15.301406 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:16:15.301597 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:16:15.304778 (+ 3181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:16:15.304787 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:16:15.304823 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:16:15.305889 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:15.305904 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.307808 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.307814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:15.311370 (+ 3556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:15.311387 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.313592 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.313602 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:15.313833 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.313879 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 01:16:15.313961 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 01:16:15.314125 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15437,"mutex_wait_us":245,"thread_start_us":17948,"threads_started":1}
I20260806 01:16:15.314831 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.286424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39872 (local address 127.25.236.130:35781)
0806 01:16:15.304578 (+ 18154us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:15.304588 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:15.304620 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:16:15.304785 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:15.304793 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:15.305017 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:15.305444 (+ 427us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:16:15.306185 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.307858 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.311607 (+ 3749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.313256 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.313473 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.313689 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:15.314279 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 01:16:15.314468 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17957,"thread_start_us":18517,"threads_started":1}
I20260806 01:16:15.314875 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.285720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:35781 (local address 127.0.0.1:39872)
0806 01:16:15.293655 (+ 7935us) negotiation.cc:107] Waiting for socket to connect
0806 01:16:15.293740 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 01:16:15.294056 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:16:15.305420 (+ 11364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:16:15.305425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:16:15.305467 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:16:15.305940 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:15.305956 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.308002 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.308010 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:15.311372 (+ 3362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:15.311387 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.313866 (+ 2479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.313873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:15.314060 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.314093 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:16:15.314191 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 01:16:15.314364 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7533,"mutex_wait_us":280,"thread_start_us":7213,"threads_started":1}
I20260806 01:16:15.315992 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.286515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45544 (local address 127.25.236.129:38249)
0806 01:16:15.303988 (+ 17473us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:15.304001 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:15.304035 (+ 34us) server_negotiation.cc:407] Connection header received
0806 01:16:15.304166 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:15.304176 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:15.304491 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:15.304759 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:16:15.306187 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.307663 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.311615 (+ 3952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.313313 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.315197 (+ 1884us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.315337 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:15.315405 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 01:16:15.315574 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17282,"thread_start_us":16708,"threads_started":1}
I20260806 01:16:15.318569 12048 tablet_service.cc:1467] Processing CreateTablet for tablet 44ac2f94bd9f44a6874908a0f829d10c (DEFAULT_TABLE table=test-table [id=0aa53bf0db6449ae92541c6db21e10bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:16:15.318589 12263 tablet_service.cc:1467] Processing CreateTablet for tablet 44ac2f94bd9f44a6874908a0f829d10c (DEFAULT_TABLE table=test-table [id=0aa53bf0db6449ae92541c6db21e10bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:16:15.319854 12048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ac2f94bd9f44a6874908a0f829d10c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:16:15.319875 12263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ac2f94bd9f44a6874908a0f829d10c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:16:15.327049 11834 tablet_service.cc:1467] Processing CreateTablet for tablet 44ac2f94bd9f44a6874908a0f829d10c (DEFAULT_TABLE table=test-table [id=0aa53bf0db6449ae92541c6db21e10bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:16:15.328954 11834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ac2f94bd9f44a6874908a0f829d10c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:16:15.339078 12384 tablet_bootstrap.cc:492] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f: Bootstrap starting.
I20260806 01:16:15.339985 12385 tablet_bootstrap.cc:492] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963: Bootstrap starting.
I20260806 01:16:15.344476 12385 tablet_bootstrap.cc:654] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963: Neither blocks nor log segments found. Creating new log.
I20260806 01:16:15.346508 12385 log.cc:824] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963: Log is configured to *not* fsync() on all Append() calls
I20260806 01:16:15.347337 12384 tablet_bootstrap.cc:654] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f: Neither blocks nor log segments found. Creating new log.
I20260806 01:16:15.349721 12385 tablet_bootstrap.cc:492] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963: No bootstrap required, opened a new log
I20260806 01:16:15.349939 12384 log.cc:824] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:16:15.350036 12385 ts_tablet_manager.cc:1397] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260806 01:16:15.353955 12384 tablet_bootstrap.cc:492] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f: No bootstrap required, opened a new log
I20260806 01:16:15.354357 12384 ts_tablet_manager.cc:1397] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.005s
I20260806 01:16:15.354938 12387 tablet_bootstrap.cc:492] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb: Bootstrap starting.
I20260806 01:16:15.359354 12387 tablet_bootstrap.cc:654] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb: Neither blocks nor log segments found. Creating new log.
I20260806 01:16:15.360359 12385 raft_consensus.cc:348] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38249 } } peers { permanent_uuid: "e4f03b223b4a41c0a550ca13e695468f" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35781 } } peers { permanent_uuid: "a60ddb6b9b38466dbad7338a7d34b963" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 39045 } }
I20260806 01:16:15.361250 12385 raft_consensus.cc:374] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:16:15.361174 12387 log.cc:824] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb: Log is configured to *not* fsync() on all Append() calls
I20260806 01:16:15.361469 12385 raft_consensus.cc:729] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a60ddb6b9b38466dbad7338a7d34b963, State: Initialized, Role: FOLLOWER
I20260806 01:16:15.362701 12385 consensus_queue.cc:260] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38249 } } peers { permanent_uuid: "e4f03b223b4a41c0a550ca13e695468f" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35781 } } peers { permanent_uuid: "a60ddb6b9b38466dbad7338a7d34b963" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 39045 } }
I20260806 01:16:15.365175 12385 ts_tablet_manager.cc:1428] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260806 01:16:15.365823 12357 heartbeater.cc:503] Master 127.25.236.190:32969 was elected leader, sending a full tablet report...
I20260806 01:16:15.366021 12387 tablet_bootstrap.cc:492] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb: No bootstrap required, opened a new log
I20260806 01:16:15.366428 12387 ts_tablet_manager.cc:1397] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260806 01:16:15.366011 12384 raft_consensus.cc:348] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38249 } } peers { permanent_uuid: "e4f03b223b4a41c0a550ca13e695468f" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35781 } } peers { permanent_uuid: "a60ddb6b9b38466dbad7338a7d34b963" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 39045 } }
I20260806 01:16:15.366767 12384 raft_consensus.cc:374] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:16:15.366897 12384 raft_consensus.cc:729] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f03b223b4a41c0a550ca13e695468f, State: Initialized, Role: FOLLOWER
I20260806 01:16:15.367771 12384 consensus_queue.cc:260] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38249 } } peers { permanent_uuid: "e4f03b223b4a41c0a550ca13e695468f" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35781 } } peers { permanent_uuid: "a60ddb6b9b38466dbad7338a7d34b963" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 39045 } }
I20260806 01:16:15.369031 12143 heartbeater.cc:503] Master 127.25.236.190:32969 was elected leader, sending a full tablet report...
I20260806 01:16:15.369781 12384 ts_tablet_manager.cc:1428] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260806 01:16:15.378278 12387 raft_consensus.cc:348] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38249 } } peers { permanent_uuid: "e4f03b223b4a41c0a550ca13e695468f" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35781 } } peers { permanent_uuid: "a60ddb6b9b38466dbad7338a7d34b963" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 39045 } }
I20260806 01:16:15.378799 12387 raft_consensus.cc:374] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:16:15.378925 12387 raft_consensus.cc:729] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6c3fc311c2c47c8b5031b453cb1d7cb, State: Initialized, Role: FOLLOWER
I20260806 01:16:15.379492 12387 consensus_queue.cc:260] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38249 } } peers { permanent_uuid: "e4f03b223b4a41c0a550ca13e695468f" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35781 } } peers { permanent_uuid: "a60ddb6b9b38466dbad7338a7d34b963" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 39045 } }
I20260806 01:16:15.380925 12387 ts_tablet_manager.cc:1428] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb: Time spent starting tablet: real 0.014s user 0.006s sys 0.006s
I20260806 01:16:15.391064 12394 raft_consensus.cc:482] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:16:15.391391 12394 raft_consensus.cc:504] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38249 } } peers { permanent_uuid: "e4f03b223b4a41c0a550ca13e695468f" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35781 } } peers { permanent_uuid: "a60ddb6b9b38466dbad7338a7d34b963" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 39045 } }
I20260806 01:16:15.392957 12394 leader_election.cc:288] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4f03b223b4a41c0a550ca13e695468f (127.25.236.130:35781), a60ddb6b9b38466dbad7338a7d34b963 (127.25.236.131:39045)
I20260806 01:16:15.402521 12399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.393526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:35781 (local address 127.25.236.129:59629)
0806 01:16:15.394631 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0806 01:16:15.394649 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:16:15.394813 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:16:15.395482 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:16:15.395486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:16:15.395506 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:16:15.395961 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:15.395974 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.397264 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.397269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:15.399259 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:15.399272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.400936 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.400948 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:15.401430 (+ 482us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.401478 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 01:16:15.401604 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 01:16:15.401768 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"mutex_wait_us":42,"thread_start_us":743,"threads_started":1}
I20260806 01:16:15.402992 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.393634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:59629 (local address 127.25.236.130:35781)
0806 01:16:15.393852 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:15.393858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:15.394767 (+ 909us) server_negotiation.cc:407] Connection header received
0806 01:16:15.395024 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:15.395029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:15.395356 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:15.395475 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:16:15.396126 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.397106 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.399423 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.400592 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.400734 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.400794 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:15.402195 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0806 01:16:15.402505 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:16:15.403569 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.393679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:43013 (local address 127.25.236.131:39045)
0806 01:16:15.393867 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:15.393874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:15.394767 (+ 893us) server_negotiation.cc:407] Connection header received
0806 01:16:15.395055 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:15.395059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:15.395123 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:15.395231 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:16:15.396360 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.397338 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.399818 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.401626 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.401784 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.401857 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:15.403063 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0806 01:16:15.403299 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:16:15.403630 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.393398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:39045 (local address 127.25.236.129:43013)
0806 01:16:15.394627 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0806 01:16:15.394649 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:16:15.394850 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:16:15.395244 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:16:15.395249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:16:15.395272 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:16:15.395718 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:15.395738 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.397511 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.397515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:15.399624 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:16:15.399642 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.402367 (+ 2725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.402377 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:16:15.402624 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.402672 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 01:16:15.403031 (+ 359us) client_negotiation.cc:240] Negotiation successful
0806 01:16:15.403207 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1067,"mutex_wait_us":20,"spinlock_wait_cycles":38656,"thread_start_us":842,"threads_started":1}
I20260806 01:16:15.404398 12076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ac2f94bd9f44a6874908a0f829d10c" candidate_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f03b223b4a41c0a550ca13e695468f" is_pre_election: true
I20260806 01:16:15.404963 12076 raft_consensus.cc:2454] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6c3fc311c2c47c8b5031b453cb1d7cb in term 0.
I20260806 01:16:15.404882 12292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ac2f94bd9f44a6874908a0f829d10c" candidate_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a60ddb6b9b38466dbad7338a7d34b963" is_pre_election: true
I20260806 01:16:15.405683 12292 raft_consensus.cc:2454] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6c3fc311c2c47c8b5031b453cb1d7cb in term 0.
I20260806 01:16:15.406049 11765 leader_election.cc:302] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e4f03b223b4a41c0a550ca13e695468f, f6c3fc311c2c47c8b5031b453cb1d7cb; no voters:
I20260806 01:16:15.406579 12394 raft_consensus.cc:2781] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:16:15.406733 12394 raft_consensus.cc:482] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:16:15.406847 12394 raft_consensus.cc:3037] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:16:15.409281 12394 raft_consensus.cc:504] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38249 } } peers { permanent_uuid: "e4f03b223b4a41c0a550ca13e695468f" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35781 } } peers { permanent_uuid: "a60ddb6b9b38466dbad7338a7d34b963" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 39045 } }
I20260806 01:16:15.409952 12394 leader_election.cc:288] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [CANDIDATE]: Term 1 election: Requested vote from peers e4f03b223b4a41c0a550ca13e695468f (127.25.236.130:35781), a60ddb6b9b38466dbad7338a7d34b963 (127.25.236.131:39045)
I20260806 01:16:15.410490 12076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ac2f94bd9f44a6874908a0f829d10c" candidate_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f03b223b4a41c0a550ca13e695468f"
I20260806 01:16:15.410584 12292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ac2f94bd9f44a6874908a0f829d10c" candidate_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a60ddb6b9b38466dbad7338a7d34b963"
I20260806 01:16:15.410722 12076 raft_consensus.cc:3037] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:16:15.410840 12292 raft_consensus.cc:3037] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:16:15.413067 12076 raft_consensus.cc:2454] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6c3fc311c2c47c8b5031b453cb1d7cb in term 1.
I20260806 01:16:15.413187 12292 raft_consensus.cc:2454] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6c3fc311c2c47c8b5031b453cb1d7cb in term 1.
I20260806 01:16:15.413484 11765 leader_election.cc:302] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e4f03b223b4a41c0a550ca13e695468f, f6c3fc311c2c47c8b5031b453cb1d7cb; no voters:
I20260806 01:16:15.413826 12394 raft_consensus.cc:2781] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:16:15.414410 12394 raft_consensus.cc:686] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [term 1 LEADER]: Becoming Leader. State: Replica: f6c3fc311c2c47c8b5031b453cb1d7cb, State: Running, Role: LEADER
I20260806 01:16:15.414951 12394 consensus_queue.cc:237] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38249 } } peers { permanent_uuid: "e4f03b223b4a41c0a550ca13e695468f" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35781 } } peers { permanent_uuid: "a60ddb6b9b38466dbad7338a7d34b963" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 39045 } }
I20260806 01:16:15.419631 11597 catalog_manager.cc:5495] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb reported cstate change: term changed from 0 to 1, leader changed from <none> to f6c3fc311c2c47c8b5031b453cb1d7cb (127.25.236.129). New cstate: current_term: 1 leader_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c3fc311c2c47c8b5031b453cb1d7cb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f03b223b4a41c0a550ca13e695468f" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a60ddb6b9b38466dbad7338a7d34b963" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 39045 } health_report { overall_health: UNKNOWN } } }
W20260806 01:16:15.443868 12358 tablet.cc:2369] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:16:15.463981 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.458420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.25.236.129:38249)
0806 01:16:15.458660 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:15.458667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:15.458688 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:16:15.458793 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:15.458797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:15.458873 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:15.459031 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:16:15.459449 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.460278 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.461787 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.462179 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.462362 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.462854 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:16:15.463360 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:16:15.463522 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:15.463635 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 01:16:15.463715 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:16:15.472481 12292 raft_consensus.cc:1261] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963 [term 1 FOLLOWER]: Refusing update from remote peer f6c3fc311c2c47c8b5031b453cb1d7cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:16:15.472545 12076 raft_consensus.cc:1261] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f [term 1 FOLLOWER]: Refusing update from remote peer f6c3fc311c2c47c8b5031b453cb1d7cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:16:15.473520 12400 consensus_queue.cc:1035] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a60ddb6b9b38466dbad7338a7d34b963" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 39045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:16:15.473861 12394 consensus_queue.cc:1035] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f03b223b4a41c0a550ca13e695468f" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:16:15.485925 12404 mvcc.cc:205] Tried to move back new op lower bound from 7315369883523727360 to 7315369883304157184. Current Snapshot: MvccSnapshot[applied={T|T < 7315369883523727360}]
I20260806 01:16:15.501744 11598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:16:15.502261 11598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:16:15.506754 11598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45dfa5c2c7ed4065a3f33212875f1c0a: Sending DeleteTablet for 3 replicas of tablet 44ac2f94bd9f44a6874908a0f829d10c
I20260806 01:16:15.508456 11834 tablet_service.cc:1514] Processing DeleteTablet for tablet 44ac2f94bd9f44a6874908a0f829d10c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:16:15 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:45544
I20260806 01:16:15.508459 12263 tablet_service.cc:1514] Processing DeleteTablet for tablet 44ac2f94bd9f44a6874908a0f829d10c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:16:15 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:51872
I20260806 01:16:15.509299 12048 tablet_service.cc:1514] Processing DeleteTablet for tablet 44ac2f94bd9f44a6874908a0f829d10c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:16:15 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:39872
I20260806 01:16:15.511945 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 11725
I20260806 01:16:15.522872 12419 tablet_replica.cc:330] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f: stopping tablet replica
I20260806 01:16:15.524833 12419 raft_consensus.cc:2229] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:16:15.526172 12419 raft_consensus.cc:2258] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:16:15.527743 12417 tablet_replica.cc:330] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963: stopping tablet replica
I20260806 01:16:15.528234 12417 raft_consensus.cc:2229] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:16:15.528741 12417 raft_consensus.cc:2258] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:16:15.529634 12418 tablet_replica.cc:330] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb: stopping tablet replica
I20260806 01:16:15.530054 12418 raft_consensus.cc:2229] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:16:15.530848 12418 raft_consensus.cc:2258] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:16:15.532905 12417 ts_tablet_manager.cc:1905] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:16:15.533017 12418 ts_tablet_manager.cc:1905] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:16:15.532996 12419 ts_tablet_manager.cc:1905] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:16:15.540750 12417 ts_tablet_manager.cc:1918] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:16:15.540984 12417 log.cc:1196] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/44ac2f94bd9f44a6874908a0f829d10c
I20260806 01:16:15.541239 12418 ts_tablet_manager.cc:1918] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:16:15.541428 12418 log.cc:1196] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/44ac2f94bd9f44a6874908a0f829d10c
I20260806 01:16:15.541909 12417 ts_tablet_manager.cc:1939] T 44ac2f94bd9f44a6874908a0f829d10c P a60ddb6b9b38466dbad7338a7d34b963: Deleting consensus metadata
I20260806 01:16:15.542078 12418 ts_tablet_manager.cc:1939] T 44ac2f94bd9f44a6874908a0f829d10c P f6c3fc311c2c47c8b5031b453cb1d7cb: Deleting consensus metadata
I20260806 01:16:15.544111 11572 catalog_manager.cc:4841] TS a60ddb6b9b38466dbad7338a7d34b963 (127.25.236.131:39045): tablet 44ac2f94bd9f44a6874908a0f829d10c (table test-table [id=0aa53bf0db6449ae92541c6db21e10bf]) successfully deleted
I20260806 01:16:15.544245 11573 catalog_manager.cc:4841] TS f6c3fc311c2c47c8b5031b453cb1d7cb (127.25.236.129:38249): tablet 44ac2f94bd9f44a6874908a0f829d10c (table test-table [id=0aa53bf0db6449ae92541c6db21e10bf]) successfully deleted
Aug 06 01:16:15 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:16:15.547552 12419 ts_tablet_manager.cc:1918] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:16:15.547765 12419 log.cc:1196] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/44ac2f94bd9f44a6874908a0f829d10c
I20260806 01:16:15.548286 12419 ts_tablet_manager.cc:1939] T 44ac2f94bd9f44a6874908a0f829d10c P e4f03b223b4a41c0a550ca13e695468f: Deleting consensus metadata
I20260806 01:16:15.550347 11575 catalog_manager.cc:4841] TS e4f03b223b4a41c0a550ca13e695468f (127.25.236.130:35781): tablet 44ac2f94bd9f44a6874908a0f829d10c (table test-table [id=0aa53bf0db6449ae92541c6db21e10bf]) successfully deleted
I20260806 01:16:15.554826 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.513251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45558 (local address 127.25.236.129:38249)
0806 01:16:15.513468 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:15.513476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:15.524025 (+ 10549us) server_negotiation.cc:407] Connection header received
0806 01:16:15.526544 (+ 2519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:15.526555 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:15.526687 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:15.526796 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:16:15.529454 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.530744 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.539627 (+ 8883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.540486 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.540598 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.547343 (+ 6745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:16:15.547360 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:16:15.547379 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:16:15.547434 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:16:15.551538 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:15.552401 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:15.552410 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:15.552412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:15.552603 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:15.553068 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:15.553073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:15.553075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:15.553429 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:16:15.553789 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:15.554143 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 01:16:15.554439 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:16:15.555590 11928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:16:15.660281 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 11725
I20260806 01:16:15.687760 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 11943
Aug 06 01:16:15 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:16:15.703297 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.688645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39876 (local address 127.25.236.130:35781)
0806 01:16:15.688815 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:15.688820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:15.688905 (+ 85us) server_negotiation.cc:407] Connection header received
0806 01:16:15.689051 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:15.689055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:15.689129 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:15.689218 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:16:15.690777 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.691559 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.692796 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.693187 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.693295 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.697554 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:16:15.697565 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:16:15.697579 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:16:15.697615 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:16:15.700371 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:15.701108 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:15.701118 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:15.701123 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:15.701262 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:15.701659 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:15.701665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:15.701668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:15.702056 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:16:15.702411 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:15.702720 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 01:16:15.702942 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 01:16:15.703943 12135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:16:15.802878 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 11943
I20260806 01:16:15.827739 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 12153
Aug 06 01:16:15 dist-test-slave-7nm7 krb5kdc[10838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:16:15.845899 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:16:15.828608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51880 (local address 127.25.236.131:39045)
0806 01:16:15.828800 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:16:15.828805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:16:15.828826 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:16:15.828930 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:16:15.828934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:16:15.829001 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:16:15.829209 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:16:15.830536 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.831308 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.832758 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:16:15.833226 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:16:15.833364 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:16:15.838508 (+ 5144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:16:15.838524 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:16:15.838541 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:16:15.838577 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:16:15.841859 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:15.842856 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:15.842865 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:15.842867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:15.842996 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:16:15.843642 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:16:15.843647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:16:15.843649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:16:15.844041 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:16:15.844590 (+ 549us) server_negotiation.cc:1035] Waiting for connection context
0806 01:16:15.844994 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 01:16:15.845399 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:16:15.846920 12347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:16:15.937965 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 12153
I20260806 01:16:15.957110 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 11546
I20260806 01:16:15.958509 11645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:16:16.071444 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 11546
I20260806 01:16:16.103171 26546 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 01:16:16.459787 26546 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 01:16:16.460433 26546 mini_ranger.cc:66] Stopping Ranger...
I20260806 01:16:16.817216 26546 mini_ranger.cc:68] Stopped Ranger
2026-08-06 01:16:16.817 UTC [10884] LOG: received smart shutdown request
2026-08-06 01:16:16.821 UTC [10884] LOG: background worker "logical replication launcher" (PID 10896) exited with exit code 1
2026-08-06 01:16:16.821 UTC [10890] LOG: shutting down
2026-08-06 01:16:16.971 UTC [10884] LOG: database system is shut down
2026-08-06T01:16:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (57331 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:16:17 dist-test-slave-7nm7 krb5kdc[12441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:16:17 dist-test-slave-7nm7 krb5kdc[12441](info): set up 2 sockets
Aug 06 01:16:17 dist-test-slave-7nm7 krb5kdc[12441](info): commencing operation
krb5kdc: starting...
W20260806 01:16:18.484166 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:16:18 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:16:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:16:18Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/rangeradmin_127.25.236.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/rangeradmin_127.25.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/rangerlookup_127.25.236.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/rangerlookup_127.25.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/HTTP_127.25.236.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/HTTP_127.25.236.148@KRBTEST.COM.keytab.
I20260806 01:16:18.919502 26546 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskVUGvte/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/postgres -l logfile start
2026-08-06 01:16:21.612 UTC [12489] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 01:16:21.612 UTC [12489] LOG: listening on IPv4 address "127.25.236.148", port 56267
2026-08-06 01:16:21.615 UTC [12489] LOG: listening on Unix socket "/tmp/.s.PGSQL.56267"
2026-08-06 01:16:21.640 UTC [12494] LOG: database system was shut down at 2026-08-06 01:16:20 UTC
2026-08-06 01:16:21.644 UTC [12489] LOG: database system is ready to accept connections
I20260806 01:16:22.888836 26546 mini_postgres.cc:96] Postgres bound to 56267
2026-08-06 01:16:22.915 UTC [12507] FATAL: database "slave" does not exist
127.25.236.148:56267 - accepting connections
I20260806 01:16:22.916091 26546 mini_ranger.cc:161] Starting Ranger...
I20260806 01:16:22.966708 26546 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 01:16:23.520054 26546 mini_ranger.cc:87] Created ranger Postgres database
I20260806 01:16:23.520231 26546 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-admin
2026-08-06 01:16:23,857 [I] DB FLAVOR :POSTGRES
2026-08-06 01:16:23,858 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 01:16:23,858 [I] Checking connection..
2026-08-06 01:16:23,858 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 01:16:24,146 [I] Checking connection passed.
2026-08-06 01:16:24,146 [I] --------- Verifying version history table ---------
2026-08-06 01:16:24,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:16:24,433 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 01:16:24,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:16:24,713 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 01:16:24,714 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 01:16:24,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 01:16:25.001 UTC [12600] WARNING: there is no transaction in progress
2026-08-06 01:16:25,009 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 01:16:25,009 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:16:25,296 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 01:16:25,297 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 01:16:25,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 01:16:25,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 01:16:25,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7nm7.c.gcp-upstream.internal','N') ;"
2026-08-06 01:16:26,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 01:16:26,419 [I] Table x_portal_user does not exist in database ranger
2026-08-06 01:16:26,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 01:16:26,718 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 01:16:26,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 01:16:26,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 01:16:27,270 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 01:16:27,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 01:16:28.308 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.330 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.351 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.370 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.391 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.411 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.471 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.478 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.494 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.505 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.516 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.531 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.542 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.549 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.556 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:28.563 UTC [12764] WARNING: there is no transaction in progress
2026-08-06 01:16:29,197 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 01:16:29,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 01:16:29,461 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 01:16:29,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 01:16:29,726 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 01:16:29,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 01:16:29,987 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 01:16:29,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 01:16:30,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 01:16:30,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7nm7.c.gcp-upstream.internal';"
2026-08-06 01:16:30,809 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 01:16:30,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 01:16:31,098 [I] DB_PATCHES have already been applied
I20260806 01:16:31.105829 26546 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-admin:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskVUGvte/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 01:16:31.105926 26546 mini_ranger.cc:193] Using host: 127.25.236.148
I20260806 01:16:31.132453 26546 mini_ranger.cc:227] Ranger admin URL: http://127.25.236.148:54637
Aug 06, 2026 1:16:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:16:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:16:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 1:16:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 1:16:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 1:16:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 1:16:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 01:16:32.584343 26546 mini_ranger.cc:160] Time spent starting Ranger: real 9.668s user 0.001s sys 0.072s
Aug 06, 2026 1:16:52 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 1:16:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 1:16:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 1:16:53 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 1:16:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 1:16:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 1:16:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 1:16:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 1:16:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 1:16:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 1:16:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 1:16:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 1:16:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 1:16:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 1:16:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 1:16:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 1:16:53 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 1:16:54 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 01:16:54.671643 26546 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/rangerkms_127.25.236.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/rangerkms_127.25.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.236.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.236.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.236.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/HTTP_127.25.236.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.236.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/HTTP_127.25.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 01:16:54 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979014, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 01:16:54.899763 26546 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 01:16:54.929605 26546 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 01:16:55.760092 26546 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 01:16:55.760229 26546 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-kms
I20260806 01:16:55.760285 26546 mini_ranger_kms.cc:227] Using postgres at 127.25.236.148:56267
2026-08-06 01:16:56,141 [I] DB FLAVOR :POSTGRES
2026-08-06 01:16:56,142 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 01:16:56,142 [I] Checking connection
2026-08-06 01:16:56,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 01:16:56,404 [I] connection success
2026-08-06 01:16:56,404 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 01:16:56,404 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 01:16:56,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 01:16:56,683 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 01:16:56,683 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 01:16:56,684 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 01:16:56,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.236.148:56267/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 01:16:56,945 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 01:16:57.470543 26546 mini_ranger_kms.cc:310] Created kms service
I20260806 01:16:57.681226 26546 mini_ranger_kms.cc:326] Created kudu user
I20260806 01:16:57.728245 26546 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 01:16:57 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979014, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:16:58.185907 26546 mini_ranger_kms.cc:384] Added ranger policy
I20260806 01:16:58.186295 26546 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-kms:/tmp/dist-test-taskVUGvte/build/asan/bin/postgresql.jar:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskVUGvte/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 01:16:58.186355 26546 mini_ranger_kms.cc:240] Using host: 127.25.236.148
I20260806 01:16:58.209829 26546 mini_ranger_kms.cc:276] Ranger KMS PID: 13051
I20260806 01:16:58.209995 26546 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.236.148:57357
Aug 06, 2026 1:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 1:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 1:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 1:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 01:17:00 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785979020, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.236.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 01:17:00 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979020, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.236.148@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:17:01.086887 26546 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.187s user 0.077s sys 0.343s
I20260806 01:17:01.087147 26546 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 01:17:01.087325 26546 mini_ranger_kms.cc:401] 127.25.236.148:57357/kms/v1/keys
Aug 06 01:17:01 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979020, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.236.148@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
Aug 06 01:17:03 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:17:03.755280 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:46335
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:46335
--ranger_config_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.236.148:57357/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:04.130254 13128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:04.130560 13128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:04.130651 13128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:04.138615 13128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:17:04.138724 13128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:04.138772 13128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:04.138827 13128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:17:04.138875 13128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:17:04.149230 13128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.236.148:57357/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:46335
--ranger_config_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:46335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:04.150509 13128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:04.151788 13128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:04.163782 13137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:04.163841 13138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:04.163802 13141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:04.164830 13128 server_base.cc:1034] running on GCE node
I20260806 01:17:04.165696 13128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:04.167256 13128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:04.168519 13128 hybrid_clock.cc:648] HybridClock initialized: now 1785979024168464 us; error 33 us; skew 500 ppm
I20260806 01:17:04.168910 13128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:04 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979024, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:04.172290 13128 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:17:04.173763 13128 webserver.cc:460] Webserver started at http://127.25.236.190:35867/ using document root <none> and password file <none>
I20260806 01:17:04.174510 13128 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:04.174623 13128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:04.175184 13128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:17:04 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979024, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:17:04.227047 13128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "86fac644bf1f4c8dbb8c11f0b87149aa"
format_stamp: "Formatted at 2026-08-06 01:17:04 on dist-test-slave-7nm7"
server_key: "f2bd75197e36f0ec3e4482af8bb5c800"
server_key_iv: "7977abd844f1cbef1c24c368b20f64d2"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:04.227983 13128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "86fac644bf1f4c8dbb8c11f0b87149aa"
format_stamp: "Formatted at 2026-08-06 01:17:04 on dist-test-slave-7nm7"
server_key: "f2bd75197e36f0ec3e4482af8bb5c800"
server_key_iv: "7977abd844f1cbef1c24c368b20f64d2"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:04.234440 13128 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260806 01:17:04.252664 13151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:04.254441 13128 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:17:04.254639 13128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "86fac644bf1f4c8dbb8c11f0b87149aa"
format_stamp: "Formatted at 2026-08-06 01:17:04 on dist-test-slave-7nm7"
server_key: "f2bd75197e36f0ec3e4482af8bb5c800"
server_key_iv: "7977abd844f1cbef1c24c368b20f64d2"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:04.254853 13128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:04.290827 13128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:04.295650 13128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:04.295991 13128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:04.316797 13128 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:46335
I20260806 01:17:04.316841 13231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:46335 every 8 connection(s)
I20260806 01:17:04.318190 13128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:17:04.322088 13232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:04.328497 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 13128
I20260806 01:17:04.328781 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:17:04.329599 26546 external_mini_cluster.cc:1428] Setting key d8975f33541cdac6146ea885a19fe22a
I20260806 01:17:04.333338 13232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa: Bootstrap starting.
I20260806 01:17:04.336491 13232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:04.337517 13232 log.cc:824] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:04.340754 13232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa: No bootstrap required, opened a new log
Aug 06 01:17:04 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:04.349200 13232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46335 } }
I20260806 01:17:04.349592 13232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:04.349702 13232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86fac644bf1f4c8dbb8c11f0b87149aa, State: Initialized, Role: FOLLOWER
I20260806 01:17:04.350458 13232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46335 } }
I20260806 01:17:04.350695 13232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:17:04.350881 13232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:17:04.351071 13232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:04.353547 13232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46335 } }
I20260806 01:17:04.353786 13236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:04.332013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49124 (local address 127.25.236.190:46335)
0806 01:17:04.332846 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:04.332858 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:04.332902 (+ 44us) server_negotiation.cc:407] Connection header received
0806 01:17:04.334923 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:04.334947 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:04.335434 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:04.335831 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:04.337247 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:04.338730 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:04.340530 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:04.341246 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:04.341455 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:04.345749 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:04.345757 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:04.345783 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:04.345836 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:04.349283 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:04.350153 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:04.350160 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:04.350168 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:04.350379 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:04.350816 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:04.350821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:04.350825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:04.351526 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:04.351871 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:04.352332 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 01:17:04.352764 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":378,"threads_started":1}
I20260806 01:17:04.354126 13232 leader_election.cc:302] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86fac644bf1f4c8dbb8c11f0b87149aa; no voters:
I20260806 01:17:04.354843 13232 leader_election.cc:288] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:17:04.355005 13238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:17:04.355757 13238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 1 LEADER]: Becoming Leader. State: Replica: 86fac644bf1f4c8dbb8c11f0b87149aa, State: Running, Role: LEADER
I20260806 01:17:04.356406 13238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46335 } }
I20260806 01:17:04.358896 13238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86fac644bf1f4c8dbb8c11f0b87149aa. Latest consensus state: current_term: 1 leader_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46335 } } }
I20260806 01:17:04.359331 13238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:04.360666 13241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46335 } } }
I20260806 01:17:04.361003 13241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:04.363550 13232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:17:04.367795 13244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:17:04.374039 13244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:17:04.374804 13232 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 01:17:04.393419 13244 catalog_manager.cc:1347] Generated new cluster ID: d1d69651d6474f679a3b5d71d8a53369
I20260806 01:17:04.393587 13244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:17:04.416210 13244 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:17:04.417615 13244 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:17:04.424790 13244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa: Generated new TSK 0
I20260806 01:17:04.425606 13244 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:17:05.294036 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.236.148:57357/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:46335
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:17:05 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785979025, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 01:17:05.790845 13296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:05.791172 13296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:05.791363 13296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:05.799750 13296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:05.799947 13296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:05.800129 13296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:17:05.816314 13296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.236.148:57357/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:46335
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:05.818277 13296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:05.820117 13296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:05.846310 13312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:05.847194 13310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:05.848740 13314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:05.848924 13296 server_base.cc:1034] running on GCE node
I20260806 01:17:05.849706 13296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:05.850996 13296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:05.852311 13296 hybrid_clock.cc:648] HybridClock initialized: now 1785979025852147 us; error 149 us; skew 500 ppm
I20260806 01:17:05.852694 13296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:05 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979025, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:05.857117 13296 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:17:05.868175 13296 webserver.cc:460] Webserver started at http://127.25.236.129:44049/ using document root <none> and password file <none>
I20260806 01:17:05.869293 13296 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:05.869464 13296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:05.869870 13296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:17:05 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979025, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:17:05.894064 13296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "882c4701da3941cb981cede16417945e"
format_stamp: "Formatted at 2026-08-06 01:17:05 on dist-test-slave-7nm7"
server_key: "5a62f68055cd9f6be3287e6ba54826bc"
server_key_iv: "43b58ae1e6fd41e9502f91cc4b9fa143"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:05.895480 13296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "882c4701da3941cb981cede16417945e"
format_stamp: "Formatted at 2026-08-06 01:17:05 on dist-test-slave-7nm7"
server_key: "5a62f68055cd9f6be3287e6ba54826bc"
server_key_iv: "43b58ae1e6fd41e9502f91cc4b9fa143"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:05.916128 13296 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.004s sys 0.011s
I20260806 01:17:05.946889 13322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:05.948843 13296 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.007s sys 0.004s
I20260806 01:17:05.949119 13296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "882c4701da3941cb981cede16417945e"
format_stamp: "Formatted at 2026-08-06 01:17:05 on dist-test-slave-7nm7"
server_key: "5a62f68055cd9f6be3287e6ba54826bc"
server_key_iv: "43b58ae1e6fd41e9502f91cc4b9fa143"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:05.949497 13296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:05.982353 13296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:05.988461 13296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:05.988870 13296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:06.000811 13296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
Aug 06 01:17:05 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979025, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:17:06.003871 13296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:06.004038 13296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 01:17:06.004213 13296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:06.004314 13296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:06.174515 13296 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:46597
I20260806 01:17:06.174539 13489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:46597 every 8 connection(s)
I20260806 01:17:06.176126 13296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:17:06.183115 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 13296
I20260806 01:17:06.183357 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:17:06.184197 26546 external_mini_cluster.cc:1428] Setting key 7048dcaa7fe7b541c90254418f620c96
Aug 06 01:17:06 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979025, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:06.220093 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:06.179579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:41597 (local address 127.25.236.190:46335)
0806 01:17:06.186826 (+ 7247us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:06.186833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:06.186857 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:17:06.186916 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:06.186920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:06.187001 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:06.187132 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:06.196166 (+ 9034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:06.197523 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:06.202985 (+ 5462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:06.203745 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:06.203861 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:06.211215 (+ 7354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:06.211221 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:06.211225 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:06.211268 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:06.215544 (+ 4276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:06.216706 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:06.216714 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:06.216720 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:06.216879 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:06.217526 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:06.217534 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:06.217540 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:06.217983 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:06.218124 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:06.219238 (+ 1114us) server_negotiation.cc:299] Negotiation successful
0806 01:17:06.219559 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7096,"thread_start_us":6902,"threads_started":1}
I20260806 01:17:06.222884 13498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:06.180408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:46335 (local address 127.25.236.129:41597)
0806 01:17:06.182888 (+ 2480us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:06.182948 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:06.185439 (+ 2491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:06.190817 (+ 5378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:06.190834 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:06.191599 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:06.192698 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:06.192730 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:06.197842 (+ 5112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:06.197851 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:06.200166 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:06.200188 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:06.204808 (+ 4620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:06.204819 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:06.205179 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:06.206693 (+ 1514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:06.206746 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:06.210894 (+ 4148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:06.215784 (+ 4890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:06.215797 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:06.215820 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:06.216538 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:06.217034 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:06.217041 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:06.217047 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:06.217379 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:06.218256 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:06.218267 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:06.218776 (+ 509us) client_negotiation.cc:769] Sending connection context
0806 01:17:06.220644 (+ 1868us) client_negotiation.cc:240] Negotiation successful
0806 01:17:06.221278 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2153,"thread_start_us":1892,"threads_started":1}
I20260806 01:17:06.225209 13490 heartbeater.cc:347] Connected to a master server at 127.25.236.190:46335
I20260806 01:17:06.225564 13490 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:06.226658 13490 heartbeater.cc:511] Master 127.25.236.190:46335 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
I20260806 01:17:06.229210 13177 ts_manager.cc:194] Registered new tserver with Master: 882c4701da3941cb981cede16417945e (127.25.236.129:46597)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
I20260806 01:17:06.231448 13177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:41597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:17:06.272071 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.236.148:57357/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:46335
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:17:06 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979025, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
W20260806 01:17:06.782019 13501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:06.782474 13501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:06.782627 13501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:06.794469 13501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:06.794620 13501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:06.794823 13501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:17:06.807825 13501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.236.148:57357/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:46335
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:06.809535 13501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:06.810928 13501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:06.825863 13519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:06.825959 13518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:06.826500 13501 server_base.cc:1034] running on GCE node
W20260806 01:17:06.826953 13521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:06.827755 13501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:06.828874 13501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:06.830101 13501 hybrid_clock.cc:648] HybridClock initialized: now 1785979026830061 us; error 33 us; skew 500 ppm
I20260806 01:17:06.830513 13501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:06 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979026, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:06.834697 13501 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:17:06.836602 13501 webserver.cc:460] Webserver started at http://127.25.236.130:38461/ using document root <none> and password file <none>
I20260806 01:17:06.837333 13501 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:06.837484 13501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:06.837880 13501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:17:06 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979026, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:17:06.855484 13501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "9f8d2468954b438b996ae64759e431bc"
format_stamp: "Formatted at 2026-08-06 01:17:06 on dist-test-slave-7nm7"
server_key: "917707fd0b76376477845225c6d8fff3"
server_key_iv: "7a1339f97b5d890a1f141dc98cd517fc"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:06.856492 13501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "9f8d2468954b438b996ae64759e431bc"
format_stamp: "Formatted at 2026-08-06 01:17:06 on dist-test-slave-7nm7"
server_key: "917707fd0b76376477845225c6d8fff3"
server_key_iv: "7a1339f97b5d890a1f141dc98cd517fc"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:06.862350 13501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:17:06.880432 13533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:06.882077 13501 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 01:17:06.882308 13501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "9f8d2468954b438b996ae64759e431bc"
format_stamp: "Formatted at 2026-08-06 01:17:06 on dist-test-slave-7nm7"
server_key: "917707fd0b76376477845225c6d8fff3"
server_key_iv: "7a1339f97b5d890a1f141dc98cd517fc"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:06.882570 13501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:06.910847 13501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:06.914999 13501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:06.915323 13501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:06.916416 13501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:06.918007 13501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:06.918083 13501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:06.918274 13501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:06.918349 13501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 01:17:06.962335 13501 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:38913
I20260806 01:17:06.962364 13720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:38913 every 8 connection(s)
I20260806 01:17:06.963819 13501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:17:06.971596 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 13501
I20260806 01:17:06.971781 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:17:06.972460 26546 external_mini_cluster.cc:1428] Setting key bb5d2dd7215c1d4e5dae780fecf2d5d9
Aug 06 01:17:06 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979026, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:06.994364 13723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:06.966762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:49481 (local address 127.25.236.190:46335)
0806 01:17:06.967358 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:06.967366 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:06.968275 (+ 909us) server_negotiation.cc:407] Connection header received
0806 01:17:06.970206 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:06.970385 (+ 179us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:06.970491 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:06.970629 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:06.973396 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:06.974766 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:06.981202 (+ 6436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:06.981653 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:06.981846 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:06.985544 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:06.985549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:06.985552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:06.985587 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:06.989045 (+ 3458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:06.990141 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:06.990147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:06.990150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:06.990330 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:06.990880 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:06.990888 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:06.990894 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:06.991654 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:06.991833 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:06.993465 (+ 1632us) server_negotiation.cc:299] Negotiation successful
0806 01:17:06.993828 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":256,"threads_started":1}
I20260806 01:17:06.995539 13724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:06.967256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:46335 (local address 127.25.236.130:49481)
0806 01:17:06.968064 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:06.968115 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:06.969805 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:06.970842 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:06.970851 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:06.971765 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:06.972980 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:06.973013 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:06.974961 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:06.974967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:06.976282 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:06.976295 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:06.981975 (+ 5680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:06.981984 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:06.982336 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:06.983103 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:06.983128 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:06.985287 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:06.989264 (+ 3977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:06.989299 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:06.989319 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:06.989947 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:06.990466 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:06.990470 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:06.990474 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:06.990717 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:06.991840 (+ 1123us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:06.991849 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:06.992668 (+ 819us) client_negotiation.cc:769] Sending connection context
0806 01:17:06.993023 (+ 355us) client_negotiation.cc:240] Negotiation successful
0806 01:17:06.994079 (+ 1056us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":347,"threads_started":1}
I20260806 01:17:06.997190 13721 heartbeater.cc:347] Connected to a master server at 127.25.236.190:46335
I20260806 01:17:06.997575 13721 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:06.998521 13721 heartbeater.cc:511] Master 127.25.236.190:46335 requested a full tablet report, sending...
I20260806 01:17:07.000334 13177 ts_manager.cc:194] Registered new tserver with Master: 9f8d2468954b438b996ae64759e431bc (127.25.236.130:38913)
I20260806 01:17:07.001407 13177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:49481
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:17:07.055760 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.236.148:57357/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:46335
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:17:07.236683 13490 heartbeater.cc:503] Master 127.25.236.190:46335 was elected leader, sending a full tablet report...
W20260806 01:17:07.406636 13727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:07.406997 13727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:07.407167 13727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:07.414614 13727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:07.414783 13727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:07.414960 13727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:17:07.425263 13727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.236.148:57357/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:46335
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:07.426828 13727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:07.428145 13727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:07.441720 13734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:07.442377 13738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:07.442530 13735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:07.444371 13727 server_base.cc:1034] running on GCE node
I20260806 01:17:07.445101 13727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:07.446470 13727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:07.447747 13727 hybrid_clock.cc:648] HybridClock initialized: now 1785979027447679 us; error 51 us; skew 500 ppm
I20260806 01:17:07.448143 13727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:07 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979027, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:07.452167 13727 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:17:07.453864 13727 webserver.cc:460] Webserver started at http://127.25.236.131:40887/ using document root <none> and password file <none>
I20260806 01:17:07.454708 13727 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:07.454906 13727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:07.455271 13727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:17:07 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979027, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:17:07.471899 13727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "1d5cf47655bd4844bb376b9c77f61e2b"
format_stamp: "Formatted at 2026-08-06 01:17:07 on dist-test-slave-7nm7"
server_key: "b6da74ed6f061af7fb2d67bc594a132f"
server_key_iv: "d46b64cc6a6b2469c344fd66d0ae85b1"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:07.473783 13727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "1d5cf47655bd4844bb376b9c77f61e2b"
format_stamp: "Formatted at 2026-08-06 01:17:07 on dist-test-slave-7nm7"
server_key: "b6da74ed6f061af7fb2d67bc594a132f"
server_key_iv: "d46b64cc6a6b2469c344fd66d0ae85b1"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:07.479938 13727 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 01:17:07.495395 13751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:07.497154 13727 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:17:07.497375 13727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "1d5cf47655bd4844bb376b9c77f61e2b"
format_stamp: "Formatted at 2026-08-06 01:17:07 on dist-test-slave-7nm7"
server_key: "b6da74ed6f061af7fb2d67bc594a132f"
server_key_iv: "d46b64cc6a6b2469c344fd66d0ae85b1"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:07.497623 13727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:07.512343 13727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:07.516507 13727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:07.516847 13727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:07.517824 13727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:07.519444 13727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:07.519551 13727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:07.519706 13727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:07.519781 13727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:07.557164 13727 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:35525
I20260806 01:17:07.557206 13924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:35525 every 8 connection(s)
I20260806 01:17:07.558580 13727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:17:07.561273 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 13727
I20260806 01:17:07.561466 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:17:07.562404 26546 external_mini_cluster.cc:1428] Setting key 9cf05ec7452c30ddd1074d9673603905
Aug 06 01:17:07 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979027, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:07.581138 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:07.561424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:45769 (local address 127.25.236.190:46335)
0806 01:17:07.562742 (+ 1318us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:07.562749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:07.563274 (+ 525us) server_negotiation.cc:407] Connection header received
0806 01:17:07.565579 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:07.565584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:07.565669 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:07.565781 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:07.567363 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:07.568096 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:07.569826 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:07.570416 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:07.570519 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:07.574494 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:07.574499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:07.574502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:07.574534 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:07.577680 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:07.578649 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:07.578653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:07.578656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:07.578861 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:07.579467 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:07.579471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:07.579473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:07.579794 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:07.579913 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:07.580679 (+ 766us) server_negotiation.cc:299] Negotiation successful
0806 01:17:07.580868 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1161,"thread_start_us":949,"threads_started":1}
I20260806 01:17:07.582304 13930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:07.562021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:46335 (local address 127.25.236.131:45769)
0806 01:17:07.563106 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:07.563144 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:07.565052 (+ 1908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:07.566079 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:07.566088 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:07.566612 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:07.567185 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:07.567202 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:07.568284 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:07.568288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:07.569680 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:07.569693 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:07.570646 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:07.570651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:07.570979 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:07.571938 (+ 959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:07.571976 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:07.574197 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:07.577818 (+ 3621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:07.577828 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:07.577847 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:07.578512 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:07.579001 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:07.579005 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:07.579008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:07.579335 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:07.579949 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:07.579956 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:07.580315 (+ 359us) client_negotiation.cc:769] Sending connection context
0806 01:17:07.580568 (+ 253us) client_negotiation.cc:240] Negotiation successful
0806 01:17:07.580980 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":607,"threads_started":1}
I20260806 01:17:07.583786 13926 heartbeater.cc:347] Connected to a master server at 127.25.236.190:46335
I20260806 01:17:07.584117 13926 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:07.584862 13926 heartbeater.cc:511] Master 127.25.236.190:46335 requested a full tablet report, sending...
I20260806 01:17:07.586498 13177 ts_manager.cc:194] Registered new tserver with Master: 1d5cf47655bd4844bb376b9c77f61e2b (127.25.236.131:35525)
I20260806 01:17:07.587392 13177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:45769
I20260806 01:17:07.591744 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:17:07.593799 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 13296
Aug 06 01:17:07 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:17:07.614421 13934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:07.595056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54206 (local address 127.25.236.129:46597)
0806 01:17:07.595743 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:07.595749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:07.595775 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:17:07.595832 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:07.595838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:07.596027 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:07.596171 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:07.598176 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:07.599428 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:07.601171 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:07.601807 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:07.601905 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:07.607609 (+ 5704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:07.607632 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:07.607654 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:07.607716 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:07.611171 (+ 3455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:07.611974 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:07.611981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:07.611984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:07.612165 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:07.612613 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:07.612617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:07.612619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:07.612996 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:07.613436 (+ 440us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:07.613711 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 01:17:07.613951 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":340,"threads_started":1}
I20260806 01:17:07.615738 13485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:07.705431 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 13296
I20260806 01:17:07.734048 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 13501
Aug 06 01:17:07 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:17:07.758339 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:07.735244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37922 (local address 127.25.236.130:38913)
0806 01:17:07.735837 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:07.735844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:07.735870 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:17:07.735946 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:07.735953 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:07.736151 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:07.736294 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:07.738071 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:07.739276 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:07.740658 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:07.741228 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:07.741341 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:07.750598 (+ 9257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:07.750615 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:07.750632 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:07.750689 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:07.755050 (+ 4361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:07.755953 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:07.755963 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:07.755965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:07.756311 (+ 346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:07.756743 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:07.756746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:07.756748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:07.757118 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:07.757366 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:07.757719 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 01:17:07.757950 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":315,"threads_started":1}
I20260806 01:17:07.759830 13714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:07.846913 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 13501
I20260806 01:17:07.871559 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 13727
Aug 06 01:17:07 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:17:07.893754 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:07.873386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48590 (local address 127.25.236.131:35525)
0806 01:17:07.874410 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:07.874418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:07.874454 (+ 36us) server_negotiation.cc:407] Connection header received
0806 01:17:07.874523 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:07.874530 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:07.874795 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:07.874942 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:07.876351 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:07.877546 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:07.879026 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:07.879631 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:07.879736 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:07.886052 (+ 6316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:07.886066 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:07.886083 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:07.886197 (+ 114us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:07.890155 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:07.890979 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:07.891000 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:07.891004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:07.891184 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:07.891800 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:07.891808 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:07.891814 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:07.892318 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:07.892727 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:07.893011 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 01:17:07.893301 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":473,"threads_started":1}
I20260806 01:17:07.895282 13916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:07.989306 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 13727
I20260806 01:17:08.012351 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 13128
I20260806 01:17:08.013777 13225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:08.122092 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 13128
I20260806 01:17:08.156808 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:46335
--webserver_interface=127.25.236.190
--webserver_port=35867
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:46335
--ranger_config_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.236.148:57357/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.236.148:57357/kms with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:08.540444 13944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:08.540800 13944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:08.540992 13944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:08.548537 13944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:17:08.548704 13944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:08.548825 13944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:08.548939 13944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:17:08.549048 13944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:17:08.559613 13944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.236.148:57357/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:46335
--ranger_config_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:46335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=35867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:08.561058 13944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:08.562569 13944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:08.575556 13951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:08.575645 13953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:08.575667 13956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:08.576668 13944 server_base.cc:1034] running on GCE node
I20260806 01:17:08.577404 13944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:08.578856 13944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:08.580084 13944 hybrid_clock.cc:648] HybridClock initialized: now 1785979028580021 us; error 56 us; skew 500 ppm
I20260806 01:17:08.580444 13944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:08 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979028, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:08.592206 13944 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:17:08.594112 13944 webserver.cc:460] Webserver started at http://127.25.236.190:35867/ using document root <none> and password file <none>
I20260806 01:17:08.595072 13944 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:08.595252 13944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 01:17:08 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979028, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:17:08.652454 13944 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:17:08.655865 13971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:08.657796 13944 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:17:08.658022 13944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "86fac644bf1f4c8dbb8c11f0b87149aa"
format_stamp: "Formatted at 2026-08-06 01:17:04 on dist-test-slave-7nm7"
server_key: "f2bd75197e36f0ec3e4482af8bb5c800"
server_key_iv: "7977abd844f1cbef1c24c368b20f64d2"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:08.659189 13944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:08.678084 13944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:08.683522 13944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:08.683877 13944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:08.706120 13944 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:46335
I20260806 01:17:08.706143 14049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:46335 every 8 connection(s)
I20260806 01:17:08.707715 13944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:17:08.708841 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 13944
I20260806 01:17:08.709878 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.236.148:57357/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:46597
--local_ip_for_outbound_sockets=127.25.236.129
--tserver_master_addrs=127.25.236.190:46335
--webserver_port=44049
--webserver_interface=127.25.236.129
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.236.148:57357/kms with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:17:08.715082 14050 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:17:08.718634 14050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa: Bootstrap starting.
I20260806 01:17:08.742903 14050 log.cc:824] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:08.752285 14050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:17:08.752861 14050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa: Bootstrap complete.
I20260806 01:17:08.764246 14050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46335 } }
I20260806 01:17:08.765389 14050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86fac644bf1f4c8dbb8c11f0b87149aa, State: Initialized, Role: FOLLOWER
I20260806 01:17:08.766177 14050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46335 } }
I20260806 01:17:08.766515 14050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:17:08.766652 14050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:17:08.766839 14050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:17:08.771461 14050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46335 } }
I20260806 01:17:08.772044 14050 leader_election.cc:302] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86fac644bf1f4c8dbb8c11f0b87149aa; no voters:
I20260806 01:17:08.772723 14050 leader_election.cc:288] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 01:17:08.772936 14056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:17:08.774060 14056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [term 2 LEADER]: Becoming Leader. State: Replica: 86fac644bf1f4c8dbb8c11f0b87149aa, State: Running, Role: LEADER
I20260806 01:17:08.775130 14056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46335 } }
I20260806 01:17:08.775584 14050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:17:08.778443 14058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46335 } } }
I20260806 01:17:08.778935 14058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:08.783512 14060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86fac644bf1f4c8dbb8c11f0b87149aa. Latest consensus state: current_term: 2 leader_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86fac644bf1f4c8dbb8c11f0b87149aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46335 } } }
I20260806 01:17:08.784438 14060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:08.788224 14065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:17:08.796074 14065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:17:08.798810 14065 catalog_manager.cc:1259] Loaded cluster ID: d1d69651d6474f679a3b5d71d8a53369
I20260806 01:17:08.799033 14065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:17:08.804702 14065 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:17:08.804862 14050 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 01:17:08.806413 14065 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa: Loaded TSK: 0
I20260806 01:17:08.808547 14065 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 01:17:09.229163 14053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:09.229700 14053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:09.229926 14053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:09.245805 14053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:09.246063 14053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:09.246332 14053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:17:09.264415 14053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.236.148:57357/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:46597
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=44049
--tserver_master_addrs=127.25.236.190:46335
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:09.266538 14053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:09.268384 14053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:09.287415 14106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:09.287803 14107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:09.289912 14112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:09.291864 14053 server_base.cc:1034] running on GCE node
I20260806 01:17:09.292807 14053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:09.300026 14053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:09.311568 14053 hybrid_clock.cc:648] HybridClock initialized: now 1785979029311486 us; error 77 us; skew 500 ppm
I20260806 01:17:09.312007 14053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:09 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979029, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:09.316637 14053 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:17:09.333428 14053 webserver.cc:460] Webserver started at http://127.25.236.129:44049/ using document root <none> and password file <none>
I20260806 01:17:09.334543 14053 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:09.334730 14053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 01:17:09 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979029, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:17:09.381618 14053 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.003s sys 0.008s
I20260806 01:17:09.385437 14127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:09.387090 14053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:17:09.387317 14053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "882c4701da3941cb981cede16417945e"
format_stamp: "Formatted at 2026-08-06 01:17:05 on dist-test-slave-7nm7"
server_key: "5a62f68055cd9f6be3287e6ba54826bc"
server_key_iv: "43b58ae1e6fd41e9502f91cc4b9fa143"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:09.388568 14053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:09.432737 14053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:09.437577 14053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:09.437989 14053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:09.439451 14053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:09.441026 14053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:09.441169 14053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:09.441323 14053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:09.441422 14053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:09.491616 14053 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:46597
I20260806 01:17:09.491714 14293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:46597 every 8 connection(s)
I20260806 01:17:09.493593 14053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:17:09.504074 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 14053
I20260806 01:17:09.505282 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.236.148:57357/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:38913
--local_ip_for_outbound_sockets=127.25.236.130
--tserver_master_addrs=127.25.236.190:46335
--webserver_port=38461
--webserver_interface=127.25.236.130
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.236.148:57357/kms with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:17:09 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979029, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:09.544067 14300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:09.503048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:46335 (local address 127.25.236.129:57925)
0806 01:17:09.517565 (+ 14517us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:09.517631 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:09.520212 (+ 2581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:09.523366 (+ 3154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:09.523378 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:09.523947 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:09.524475 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:09.524492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:09.526095 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:09.526101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:09.528625 (+ 2524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:09.528642 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:09.530508 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:09.530516 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:09.530712 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:09.531517 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:09.531544 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:09.534026 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:09.538902 (+ 4876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:09.538913 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:09.538932 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:09.539458 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:09.540020 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:09.540026 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:09.540030 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:09.540247 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:09.541157 (+ 910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:09.541167 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:09.541549 (+ 382us) client_negotiation.cc:769] Sending connection context
0806 01:17:09.541854 (+ 305us) client_negotiation.cc:240] Negotiation successful
0806 01:17:09.542545 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14166,"thread_start_us":17518,"threads_started":1}
I20260806 01:17:09.544061 14299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:09.502904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:57925 (local address 127.25.236.190:46335)
0806 01:17:09.521894 (+ 18990us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:09.521912 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:09.521959 (+ 47us) server_negotiation.cc:407] Connection header received
0806 01:17:09.522427 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:09.522452 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:09.522847 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:09.523207 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:09.524705 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:09.525919 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:09.528825 (+ 2906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:09.529887 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:09.530124 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:09.534365 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:09.534374 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:09.534389 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:09.534434 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:09.538683 (+ 4249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:09.539659 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:09.539668 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:09.539673 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:09.539850 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:09.540391 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:09.540397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:09.540402 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:09.540990 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:09.541237 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:09.542101 (+ 864us) server_negotiation.cc:299] Negotiation successful
0806 01:17:09.542556 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18718,"thread_start_us":18490,"threads_started":1}
I20260806 01:17:09.548190 14297 heartbeater.cc:347] Connected to a master server at 127.25.236.190:46335
I20260806 01:17:09.548591 14297 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:09.549369 14297 heartbeater.cc:511] Master 127.25.236.190:46335 requested a full tablet report, sending...
W20260806 01:17:09.553280 14297 heartbeater.cc:644] Failed to heartbeat to 127.25.236.190:46335 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 01:17:09.553547 14297 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:09.553862 14297 heartbeater.cc:511] Master 127.25.236.190:46335 requested a full tablet report, sending...
I20260806 01:17:09.555078 14297 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:09.555663 14297 heartbeater.cc:511] Master 127.25.236.190:46335 requested a full tablet report, sending...
W20260806 01:17:09.556906 14297 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:17:10.013010 14301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:10.013306 14301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:10.013461 14301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:10.021893 14301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:10.022063 14301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:10.022323 14301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:17:10.033751 14301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.236.148:57357/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:38913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=38461
--tserver_master_addrs=127.25.236.190:46335
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:10.035488 14301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:10.036875 14301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:10.053711 14332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:10.053987 14301 server_base.cc:1034] running on GCE node
W20260806 01:17:10.054535 14335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:10.054737 14331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:10.056137 14301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:10.058046 14301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:10.059376 14301 hybrid_clock.cc:648] HybridClock initialized: now 1785979030059305 us; error 55 us; skew 500 ppm
I20260806 01:17:10.059919 14301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:10 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979030, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:10.065039 14301 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:17:10.082123 14301 webserver.cc:460] Webserver started at http://127.25.236.130:38461/ using document root <none> and password file <none>
I20260806 01:17:10.083390 14301 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:10.083673 14301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 01:17:10 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979030, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:17:10.130900 14301 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.003s sys 0.009s
I20260806 01:17:10.146543 14349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:10.148751 14301 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.015s sys 0.000s
I20260806 01:17:10.148978 14301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "9f8d2468954b438b996ae64759e431bc"
format_stamp: "Formatted at 2026-08-06 01:17:06 on dist-test-slave-7nm7"
server_key: "917707fd0b76376477845225c6d8fff3"
server_key_iv: "7a1339f97b5d890a1f141dc98cd517fc"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:10.150480 14301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:10.183761 14301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:10.190889 14301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:10.191360 14301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:10.192746 14301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:10.194746 14301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:10.194865 14301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:10.195000 14301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:10.195086 14301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:17:10 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785979030, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:10.289592 14301 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:38913
I20260806 01:17:10.289590 14522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:38913 every 8 connection(s)
I20260806 01:17:10.291239 14301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:17:10.293936 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 14301
I20260806 01:17:10.295059 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.236.148:57357/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:35525
--local_ip_for_outbound_sockets=127.25.236.131
--tserver_master_addrs=127.25.236.190:46335
--webserver_port=40887
--webserver_interface=127.25.236.131
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.236.148:57357/kms with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:17:10 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979030, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:10.337281 14530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:10.295610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:46335 (local address 127.25.236.130:53197)
0806 01:17:10.311193 (+ 15583us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:10.311275 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:10.313955 (+ 2680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:10.315114 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:10.315129 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:10.315879 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:10.316750 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:10.316778 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:10.318481 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:10.318488 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:10.320625 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:10.320671 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:10.322114 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:10.322123 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:10.322580 (+ 457us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:10.323436 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:10.323472 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:10.326961 (+ 3489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:10.331897 (+ 4936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:10.331908 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:10.331928 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:10.332583 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:10.333071 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:10.333077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:10.333080 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:10.333351 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:10.334154 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:10.334164 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:10.334704 (+ 540us) client_negotiation.cc:769] Sending connection context
0806 01:17:10.335030 (+ 326us) client_negotiation.cc:240] Negotiation successful
0806 01:17:10.335645 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15151,"thread_start_us":14861,"threads_started":1}
I20260806 01:17:10.338567 14529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:10.294931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:53197 (local address 127.25.236.190:46335)
0806 01:17:10.312266 (+ 17335us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:10.312276 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:10.312304 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:17:10.314391 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:10.314400 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:10.314502 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:10.314623 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:10.317003 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:10.318211 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:10.320871 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:10.321593 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:10.321736 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:10.327286 (+ 5550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:10.327294 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:10.327299 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:10.327348 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:10.331694 (+ 4346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:10.332756 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:10.332765 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:10.332770 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:10.332921 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:10.333502 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:10.333508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:10.333511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:10.333885 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:10.334038 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:10.337833 (+ 3795us) server_negotiation.cc:299] Negotiation successful
0806 01:17:10.338103 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17169,"thread_start_us":20040,"threads_started":1}
I20260806 01:17:10.341686 14526 heartbeater.cc:347] Connected to a master server at 127.25.236.190:46335
I20260806 01:17:10.342041 14526 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:10.342967 14526 heartbeater.cc:511] Master 127.25.236.190:46335 requested a full tablet report, sending...
I20260806 01:17:10.345089 13993 ts_manager.cc:194] Registered new tserver with Master: 9f8d2468954b438b996ae64759e431bc (127.25.236.130:38913)
I20260806 01:17:10.346936 13993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:53197
I20260806 01:17:10.557075 14297 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:10.557617 14297 heartbeater.cc:511] Master 127.25.236.190:46335 requested a full tablet report, sending...
I20260806 01:17:10.559041 13993 ts_manager.cc:194] Registered new tserver with Master: 882c4701da3941cb981cede16417945e (127.25.236.129:46597)
I20260806 01:17:10.560416 13993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:57925
Aug 06 01:17:10 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979030, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
W20260806 01:17:10.743518 14531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:10.744027 14531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:10.744244 14531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:10.757733 14531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:10.757930 14531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:10.758168 14531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:17:10.774991 14531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.236.148:57357/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:35525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=40887
--tserver_master_addrs=127.25.236.190:46335
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:10.776610 14531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:10.778180 14531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:10.797601 14545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:10.797557 14546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:10.806843 14531 server_base.cc:1034] running on GCE node
W20260806 01:17:10.806972 14550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:10.807811 14531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:10.808959 14531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:10.810351 14531 hybrid_clock.cc:648] HybridClock initialized: now 1785979030810274 us; error 56 us; skew 500 ppm
I20260806 01:17:10.810731 14531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:10 dist-test-slave-7nm7 krb5kdc[12441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979030, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:10.814440 14531 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:17:10.816397 14531 webserver.cc:460] Webserver started at http://127.25.236.131:40887/ using document root <none> and password file <none>
I20260806 01:17:10.817237 14531 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:10.817438 14531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 01:17:10 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979030, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:17:10.841132 14531 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.002s
I20260806 01:17:10.845425 14563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:10.847169 14531 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 01:17:10.847380 14531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "1d5cf47655bd4844bb376b9c77f61e2b"
format_stamp: "Formatted at 2026-08-06 01:17:07 on dist-test-slave-7nm7"
server_key: "b6da74ed6f061af7fb2d67bc594a132f"
server_key_iv: "d46b64cc6a6b2469c344fd66d0ae85b1"
server_key_version: "kuduclusterkey@0"
I20260806 01:17:10.848608 14531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:10.870739 14531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:10.875722 14531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:10.876065 14531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:10.877386 14531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:10.879271 14531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:10.879390 14531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:10.879522 14531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:10.879607 14531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:10.930109 14531 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:35525
I20260806 01:17:10.930194 14735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:35525 every 8 connection(s)
I20260806 01:17:10.932220 14531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:17:10.934484 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 14531
Aug 06 01:17:10 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979030, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:10.991683 14743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:10.937932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49130 (local address 127.25.236.190:46335)
0806 01:17:10.962458 (+ 24526us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:10.962467 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:10.971034 (+ 8567us) server_negotiation.cc:407] Connection header received
0806 01:17:10.971404 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:10.971412 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:10.971511 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:10.971643 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:10.974597 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:10.976169 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:10.978152 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:10.978802 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:10.978892 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:10.982703 (+ 3811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:10.982711 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:10.982717 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:10.982764 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:10.987075 (+ 4311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:10.988019 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:10.988028 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:10.988033 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:10.988257 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:10.988783 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:10.988788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:10.988793 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:10.989547 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:10.990267 (+ 720us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:10.990951 (+ 684us) server_negotiation.cc:299] Negotiation successful
0806 01:17:10.991258 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24371,"thread_start_us":24733,"threads_started":1}
I20260806 01:17:10.992930 14747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:10.938251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:56479 (local address 127.25.236.190:46335)
0806 01:17:10.962676 (+ 24425us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:10.962683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:10.963985 (+ 1302us) server_negotiation.cc:407] Connection header received
0806 01:17:10.967338 (+ 3353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:10.967348 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:10.967481 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:10.967585 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:10.971284 (+ 3699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:10.973061 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:10.977307 (+ 4246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:10.977976 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:10.978113 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:10.983134 (+ 5021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:10.983141 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:10.983145 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:10.983207 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:10.987924 (+ 4717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:10.988901 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:10.988905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:10.988908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:10.989013 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:10.990267 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:10.990273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:10.990277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:10.990660 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:10.990836 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:10.992311 (+ 1475us) server_negotiation.cc:299] Negotiation successful
0806 01:17:10.992531 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24292,"thread_start_us":23669,"threads_started":1}
I20260806 01:17:10.996023 14741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:10.937180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:46335 (local address 127.25.236.131:56479)
0806 01:17:10.963161 (+ 25981us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:10.963233 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:10.966885 (+ 3652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:10.968314 (+ 1429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:10.968331 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:10.969142 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:10.970207 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:10.970261 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:10.973673 (+ 3412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:10.973682 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:10.977099 (+ 3417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:10.977126 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:10.978496 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:10.978504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:10.978832 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:10.979733 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:10.979768 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:10.982833 (+ 3065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:10.988086 (+ 5253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:10.988098 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:10.988115 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:10.988767 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:10.989315 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:10.989324 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:10.989332 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:10.989833 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:10.990932 (+ 1099us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:10.990942 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:10.991442 (+ 500us) client_negotiation.cc:769] Sending connection context
0806 01:17:10.993408 (+ 1966us) client_negotiation.cc:240] Negotiation successful
0806 01:17:10.994049 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25574,"thread_start_us":25463,"threads_started":1}
I20260806 01:17:10.997422 14738 heartbeater.cc:347] Connected to a master server at 127.25.236.190:46335
I20260806 01:17:10.997758 14738 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:10.998592 14738 heartbeater.cc:511] Master 127.25.236.190:46335 requested a full tablet report, sending...
I20260806 01:17:11.000032 13993 ts_manager.cc:194] Registered new tserver with Master: 1d5cf47655bd4844bb376b9c77f61e2b (127.25.236.131:35525)
I20260806 01:17:11.000844 13993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:56479
I20260806 01:17:11.010010 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:17:11.027778 14747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.013733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49138 (local address 127.25.236.190:46335)
0806 01:17:11.013929 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:11.013938 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:11.014497 (+ 559us) server_negotiation.cc:407] Connection header received
0806 01:17:11.014702 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:11.014708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:11.014787 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:11.014894 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:11.016451 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.017191 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.018625 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.019056 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.019205 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.021789 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:11.021795 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:11.021800 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:11.021841 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:11.024771 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:11.025511 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:11.025518 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:11.025522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:11.025656 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:11.026035 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:11.026040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:11.026044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:11.026544 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:11.026818 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:11.027168 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 01:17:11.027430 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:17:11.033072 13993 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:17:11.043826 13993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:17:11.105907 14763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.072660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46597 (local address 127.0.0.1:54222)
0806 01:17:11.094258 (+ 21598us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:11.094280 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:11.094407 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:11.096149 (+ 1742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:11.096158 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:11.096193 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:11.096651 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.096664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.099050 (+ 2386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.099059 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.101885 (+ 2826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.101900 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.104712 (+ 2812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.104722 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.105042 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.105108 (+ 66us) client_negotiation.cc:769] Sending connection context
0806 01:17:11.105268 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 01:17:11.105497 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21440,"thread_start_us":21304,"threads_started":1}
Aug 06 01:17:11 dist-test-slave-7nm7 krb5kdc[12441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979030, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for HTTP/127.25.236.148@KRBTEST.COM
I20260806 01:17:11.106208 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.073375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54222 (local address 127.25.236.129:46597)
0806 01:17:11.095482 (+ 22107us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:11.095492 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:11.095523 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:17:11.095628 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:11.095637 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:11.095917 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:11.096130 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:11.096887 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.098869 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.102126 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.104057 (+ 1931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.104286 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.104526 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:11.105475 (+ 949us) server_negotiation.cc:299] Negotiation successful
0806 01:17:11.105696 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21920,"thread_start_us":18383,"threads_started":1}
I20260806 01:17:11.111186 14202 tablet_service.cc:1467] Processing CreateTablet for tablet c7f29b6598e6423ebe9b40dc55760a47 (DEFAULT_TABLE table=test-table [id=c3b5acdd3c6c4633b7ad580f6bbeb12b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:17:11.115442 14202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7f29b6598e6423ebe9b40dc55760a47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:11.127566 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.072615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:35525 (local address 127.0.0.1:48600)
0806 01:17:11.091076 (+ 18461us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:11.091124 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:11.091314 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:11.119347 (+ 28033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:11.119353 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:11.119382 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:11.119769 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.119782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.122088 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.122094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.124348 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.124366 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.126604 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.126612 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.126804 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.126847 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 01:17:11.126921 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 01:17:11.127078 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18162,"thread_start_us":24361,"threads_started":1}
I20260806 01:17:11.128875 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.073599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.25.236.131:35525)
0806 01:17:11.118712 (+ 45113us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:11.118722 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:11.118759 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:17:11.118865 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:11.118872 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:11.119106 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:11.120011 (+ 905us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:11.120316 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.121876 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.124598 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.126313 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.127932 (+ 1619us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.128256 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:11.128349 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 01:17:11.128535 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44919,"threads_started":1}
I20260806 01:17:11.133419 14635 tablet_service.cc:1467] Processing CreateTablet for tablet c7f29b6598e6423ebe9b40dc55760a47 (DEFAULT_TABLE table=test-table [id=c3b5acdd3c6c4633b7ad580f6bbeb12b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:17:11.135354 14764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.084560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:38913 (local address 127.0.0.1:37932)
0806 01:17:11.097227 (+ 12667us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:11.097252 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:11.097358 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:11.116188 (+ 18830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:11.116194 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:11.116227 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:11.116759 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.116775 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.127254 (+ 10479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.127260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.129502 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.129522 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.134511 (+ 4989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.134518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.134717 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.134753 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 01:17:11.134854 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 01:17:11.134998 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12521,"thread_start_us":6077,"threads_started":1}
I20260806 01:17:11.136183 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.085011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37932 (local address 127.25.236.130:38913)
0806 01:17:11.115111 (+ 30100us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:11.115121 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:11.115157 (+ 36us) server_negotiation.cc:407] Connection header received
0806 01:17:11.115276 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:11.115284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:11.115679 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:11.117010 (+ 1331us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:11.117291 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.127034 (+ 9743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.130485 (+ 3451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.131926 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.132107 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.132278 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:11.135671 (+ 3393us) server_negotiation.cc:299] Negotiation successful
0806 01:17:11.135867 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29898,"thread_start_us":30848,"threads_started":1}
I20260806 01:17:11.140328 14431 tablet_service.cc:1467] Processing CreateTablet for tablet c7f29b6598e6423ebe9b40dc55760a47 (DEFAULT_TABLE table=test-table [id=c3b5acdd3c6c4633b7ad580f6bbeb12b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:17:11.141860 14431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7f29b6598e6423ebe9b40dc55760a47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:11.143142 14635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7f29b6598e6423ebe9b40dc55760a47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:11.174556 14770 tablet_bootstrap.cc:492] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e: Bootstrap starting.
I20260806 01:17:11.176872 14772 tablet_bootstrap.cc:492] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b: Bootstrap starting.
I20260806 01:17:11.179774 14770 tablet_bootstrap.cc:654] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:11.181555 14770 log.cc:824] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:11.181830 14771 tablet_bootstrap.cc:492] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc: Bootstrap starting.
I20260806 01:17:11.187563 14772 tablet_bootstrap.cc:654] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:11.190790 14771 tablet_bootstrap.cc:654] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:11.192159 14772 log.cc:824] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:11.193955 14771 log.cc:824] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:11.195374 14770 tablet_bootstrap.cc:492] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e: No bootstrap required, opened a new log
I20260806 01:17:11.195861 14770 ts_tablet_manager.cc:1397] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.008s
I20260806 01:17:11.211566 14772 tablet_bootstrap.cc:492] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b: No bootstrap required, opened a new log
I20260806 01:17:11.211943 14770 raft_consensus.cc:348] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35525 } } peers { permanent_uuid: "882c4701da3941cb981cede16417945e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46597 } } peers { permanent_uuid: "9f8d2468954b438b996ae64759e431bc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38913 } }
I20260806 01:17:11.212472 14770 raft_consensus.cc:374] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:11.212608 14770 raft_consensus.cc:729] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 882c4701da3941cb981cede16417945e, State: Initialized, Role: FOLLOWER
I20260806 01:17:11.213446 14770 consensus_queue.cc:260] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35525 } } peers { permanent_uuid: "882c4701da3941cb981cede16417945e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46597 } } peers { permanent_uuid: "9f8d2468954b438b996ae64759e431bc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38913 } }
I20260806 01:17:11.214607 14772 ts_tablet_manager.cc:1397] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b: Time spent bootstrapping tablet: real 0.038s user 0.006s sys 0.010s
I20260806 01:17:11.216392 14771 tablet_bootstrap.cc:492] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc: No bootstrap required, opened a new log
I20260806 01:17:11.226635 14771 ts_tablet_manager.cc:1397] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc: Time spent bootstrapping tablet: real 0.045s user 0.013s sys 0.003s
I20260806 01:17:11.232360 14770 ts_tablet_manager.cc:1428] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e: Time spent starting tablet: real 0.036s user 0.009s sys 0.009s
I20260806 01:17:11.234454 14297 heartbeater.cc:503] Master 127.25.236.190:46335 was elected leader, sending a full tablet report...
I20260806 01:17:11.234709 14772 raft_consensus.cc:348] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35525 } } peers { permanent_uuid: "882c4701da3941cb981cede16417945e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46597 } } peers { permanent_uuid: "9f8d2468954b438b996ae64759e431bc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38913 } }
I20260806 01:17:11.235201 14772 raft_consensus.cc:374] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:11.235307 14772 raft_consensus.cc:729] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5cf47655bd4844bb376b9c77f61e2b, State: Initialized, Role: FOLLOWER
I20260806 01:17:11.236245 14772 consensus_queue.cc:260] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35525 } } peers { permanent_uuid: "882c4701da3941cb981cede16417945e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46597 } } peers { permanent_uuid: "9f8d2468954b438b996ae64759e431bc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38913 } }
I20260806 01:17:11.238353 14771 raft_consensus.cc:348] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35525 } } peers { permanent_uuid: "882c4701da3941cb981cede16417945e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46597 } } peers { permanent_uuid: "9f8d2468954b438b996ae64759e431bc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38913 } }
I20260806 01:17:11.238901 14771 raft_consensus.cc:374] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:11.239050 14771 raft_consensus.cc:729] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f8d2468954b438b996ae64759e431bc, State: Initialized, Role: FOLLOWER
I20260806 01:17:11.239835 14771 consensus_queue.cc:260] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35525 } } peers { permanent_uuid: "882c4701da3941cb981cede16417945e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46597 } } peers { permanent_uuid: "9f8d2468954b438b996ae64759e431bc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38913 } }
I20260806 01:17:11.247254 14738 heartbeater.cc:503] Master 127.25.236.190:46335 was elected leader, sending a full tablet report...
I20260806 01:17:11.248080 14772 ts_tablet_manager.cc:1428] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b: Time spent starting tablet: real 0.032s user 0.012s sys 0.011s
W20260806 01:17:11.251652 14298 tablet.cc:2369] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:17:11.257180 14771 ts_tablet_manager.cc:1428] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc: Time spent starting tablet: real 0.030s user 0.016s sys 0.004s
I20260806 01:17:11.257627 14526 heartbeater.cc:503] Master 127.25.236.190:46335 was elected leader, sending a full tablet report...
W20260806 01:17:11.294649 14528 tablet.cc:2369] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:17:11.440580 14739 tablet.cc:2369] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:17:11.518043 14780 raft_consensus.cc:482] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:17:11.518540 14780 raft_consensus.cc:504] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35525 } } peers { permanent_uuid: "882c4701da3941cb981cede16417945e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46597 } } peers { permanent_uuid: "9f8d2468954b438b996ae64759e431bc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38913 } }
I20260806 01:17:11.520439 14780 leader_election.cc:288] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 882c4701da3941cb981cede16417945e (127.25.236.129:46597), 9f8d2468954b438b996ae64759e431bc (127.25.236.130:38913)
I20260806 01:17:11.520547 14779 raft_consensus.cc:482] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:17:11.520854 14779 raft_consensus.cc:504] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35525 } } peers { permanent_uuid: "882c4701da3941cb981cede16417945e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46597 } } peers { permanent_uuid: "9f8d2468954b438b996ae64759e431bc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38913 } }
I20260806 01:17:11.523098 14779 leader_election.cc:288] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d5cf47655bd4844bb376b9c77f61e2b (127.25.236.131:35525), 9f8d2468954b438b996ae64759e431bc (127.25.236.130:38913)
I20260806 01:17:11.542910 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.521600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:34077 (local address 127.25.236.129:46597)
0806 01:17:11.522068 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:11.522075 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:11.532926 (+ 10851us) server_negotiation.cc:407] Connection header received
0806 01:17:11.533448 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:11.533458 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:11.533568 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:11.533644 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:11.534610 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.535967 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.539402 (+ 3435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.541263 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.541453 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.541573 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:11.542359 (+ 786us) server_negotiation.cc:299] Negotiation successful
0806 01:17:11.542550 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297}
I20260806 01:17:11.543838 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.520649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46597 (local address 127.25.236.131:34077)
0806 01:17:11.532784 (+ 12135us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:11.532812 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:11.533248 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:11.533891 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:11.533897 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:11.533939 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:11.534392 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.534407 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.536194 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.536202 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.538976 (+ 2774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.538997 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.541776 (+ 2779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.541785 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.542085 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.542142 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 01:17:11.543149 (+ 1007us) client_negotiation.cc:240] Negotiation successful
0806 01:17:11.543339 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11957,"threads_started":1}
I20260806 01:17:11.552733 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.521817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:51689 (local address 127.25.236.130:38913)
0806 01:17:11.522271 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:11.522278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:11.544223 (+ 21945us) server_negotiation.cc:407] Connection header received
0806 01:17:11.544550 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:11.544560 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:11.544688 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:11.544792 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:11.545540 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.546906 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.549775 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.551219 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.551357 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.551513 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:11.552151 (+ 638us) server_negotiation.cc:299] Negotiation successful
0806 01:17:11.552356 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":287}
I20260806 01:17:11.553601 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.520941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:38913 (local address 127.25.236.131:51689)
0806 01:17:11.544091 (+ 23150us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:11.544118 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:11.544359 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:11.545059 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:11.545067 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:11.545103 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:11.545348 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.545361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.547193 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.547202 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.549550 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.549570 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.551664 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.551670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.551916 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.551955 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 01:17:11.553021 (+ 1066us) client_negotiation.cc:240] Negotiation successful
0806 01:17:11.553195 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23043,"thread_start_us":12002,"threads_started":1}
I20260806 01:17:11.554808 14461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f29b6598e6423ebe9b40dc55760a47" candidate_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f8d2468954b438b996ae64759e431bc" is_pre_election: true
I20260806 01:17:11.555527 14461 raft_consensus.cc:2454] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d5cf47655bd4844bb376b9c77f61e2b in term 0.
I20260806 01:17:11.556514 14567 leader_election.cc:302] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d5cf47655bd4844bb376b9c77f61e2b, 9f8d2468954b438b996ae64759e431bc; no voters:
I20260806 01:17:11.557036 14780 raft_consensus.cc:2781] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:17:11.557196 14780 raft_consensus.cc:482] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:17:11.557351 14780 raft_consensus.cc:3037] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:11.560127 14798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.523665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:38913 (local address 127.25.236.129:56031)
0806 01:17:11.529695 (+ 6030us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:11.529722 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:11.529871 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:11.551303 (+ 21432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:11.551308 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:11.551343 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:11.551711 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.551722 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.553311 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.553316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.555533 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.555553 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.559303 (+ 3750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.559311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.559501 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.559537 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 01:17:11.559609 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 01:17:11.559743 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5899,"mutex_wait_us":21,"thread_start_us":5731,"threads_started":1}
I20260806 01:17:11.560949 14799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.524136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:56031 (local address 127.25.236.130:38913)
0806 01:17:11.550936 (+ 26800us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:11.550943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:11.550973 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:17:11.551051 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:11.551057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:11.551145 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:11.551276 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:11.551915 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.553144 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.555695 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.559027 (+ 3332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.560407 (+ 1380us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.560473 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:11.560522 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 01:17:11.560646 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":26650,"thread_start_us":31049,"threads_started":1}
I20260806 01:17:11.560705 14234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f29b6598e6423ebe9b40dc55760a47" candidate_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882c4701da3941cb981cede16417945e" is_pre_election: true
I20260806 01:17:11.561239 14234 raft_consensus.cc:2454] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d5cf47655bd4844bb376b9c77f61e2b in term 0.
I20260806 01:17:11.561280 14780 raft_consensus.cc:504] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35525 } } peers { permanent_uuid: "882c4701da3941cb981cede16417945e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46597 } } peers { permanent_uuid: "9f8d2468954b438b996ae64759e431bc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38913 } }
I20260806 01:17:11.562651 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.525662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:56565 (local address 127.25.236.131:35525)
0806 01:17:11.525906 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:11.525915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:11.550052 (+ 24137us) server_negotiation.cc:407] Connection header received
0806 01:17:11.551860 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:11.551866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:11.551941 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:11.552008 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:11.553105 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.554151 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.558519 (+ 4368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.559970 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.560117 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.560212 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:11.562304 (+ 2092us) server_negotiation.cc:299] Negotiation successful
0806 01:17:11.562437 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 01:17:11.562731 14459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f29b6598e6423ebe9b40dc55760a47" candidate_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f8d2468954b438b996ae64759e431bc"
I20260806 01:17:11.563002 14234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f29b6598e6423ebe9b40dc55760a47" candidate_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882c4701da3941cb981cede16417945e"
I20260806 01:17:11.563061 14459 raft_consensus.cc:3037] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:11.563257 14234 raft_consensus.cc:3037] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:11.563445 14804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.525012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:35525 (local address 127.25.236.129:56565)
0806 01:17:11.549941 (+ 24929us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:11.549964 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:11.551665 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:11.552218 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:11.552223 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:11.552249 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:11.552670 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.552683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.554604 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.554609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.558289 (+ 3680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:11.558307 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.561917 (+ 3610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.561923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:11.562084 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.562114 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 01:17:11.562807 (+ 693us) client_negotiation.cc:240] Negotiation successful
0806 01:17:11.562943 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24776,"mutex_wait_us":97,"thread_start_us":25046,"threads_started":1}
I20260806 01:17:11.564815 14461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f29b6598e6423ebe9b40dc55760a47" candidate_uuid: "882c4701da3941cb981cede16417945e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f8d2468954b438b996ae64759e431bc" is_pre_election: true
I20260806 01:17:11.565922 14780 leader_election.cc:288] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [CANDIDATE]: Term 1 election: Requested vote from peers 882c4701da3941cb981cede16417945e (127.25.236.129:46597), 9f8d2468954b438b996ae64759e431bc (127.25.236.130:38913)
I20260806 01:17:11.567181 14459 raft_consensus.cc:2454] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d5cf47655bd4844bb376b9c77f61e2b in term 1.
I20260806 01:17:11.567009 14659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f29b6598e6423ebe9b40dc55760a47" candidate_uuid: "882c4701da3941cb981cede16417945e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" is_pre_election: true
I20260806 01:17:11.567596 14659 raft_consensus.cc:2379] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 882c4701da3941cb981cede16417945e in current term 1: Already voted for candidate 1d5cf47655bd4844bb376b9c77f61e2b in this term.
I20260806 01:17:11.567673 14567 leader_election.cc:302] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d5cf47655bd4844bb376b9c77f61e2b, 9f8d2468954b438b996ae64759e431bc; no voters:
I20260806 01:17:11.568115 14780 raft_consensus.cc:2781] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:17:11.568492 14234 raft_consensus.cc:2454] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d5cf47655bd4844bb376b9c77f61e2b in term 1.
I20260806 01:17:11.568629 14133 leader_election.cc:302] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 882c4701da3941cb981cede16417945e; no voters: 1d5cf47655bd4844bb376b9c77f61e2b, 9f8d2468954b438b996ae64759e431bc
I20260806 01:17:11.569053 14780 raft_consensus.cc:686] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 1 LEADER]: Becoming Leader. State: Replica: 1d5cf47655bd4844bb376b9c77f61e2b, State: Running, Role: LEADER
I20260806 01:17:11.569128 14779 raft_consensus.cc:2726] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 01:17:11.569653 14780 consensus_queue.cc:237] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35525 } } peers { permanent_uuid: "882c4701da3941cb981cede16417945e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46597 } } peers { permanent_uuid: "9f8d2468954b438b996ae64759e431bc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38913 } }
I20260806 01:17:11.575095 13993 catalog_manager.cc:5495] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d5cf47655bd4844bb376b9c77f61e2b (127.25.236.131). New cstate: current_term: 1 leader_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5cf47655bd4844bb376b9c77f61e2b" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "882c4701da3941cb981cede16417945e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f8d2468954b438b996ae64759e431bc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38913 } health_report { overall_health: UNKNOWN } } }
I20260806 01:17:11.670220 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.662976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.25.236.131:35525)
0806 01:17:11.663166 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:11.663172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:11.663580 (+ 408us) server_negotiation.cc:407] Connection header received
0806 01:17:11.663847 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:11.663852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:11.663961 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:11.664125 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:17:11.664569 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.665752 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.667524 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.668251 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.668501 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.669041 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:17:11.669569 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:17:11.669735 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:11.669821 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 01:17:11.669907 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:17:11.680256 14459 raft_consensus.cc:1261] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc [term 1 FOLLOWER]: Refusing update from remote peer 1d5cf47655bd4844bb376b9c77f61e2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:17:11.680298 14234 raft_consensus.cc:1261] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [term 1 FOLLOWER]: Refusing update from remote peer 1d5cf47655bd4844bb376b9c77f61e2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:17:11.681313 14805 consensus_queue.cc:1035] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f8d2468954b438b996ae64759e431bc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:17:11.681723 14780 consensus_queue.cc:1035] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "882c4701da3941cb981cede16417945e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:17:11.732920 13993 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:17:11.733227 13993 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:17:11.737839 13993 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86fac644bf1f4c8dbb8c11f0b87149aa: Sending DeleteTablet for 3 replicas of tablet c7f29b6598e6423ebe9b40dc55760a47
I20260806 01:17:11.739151 14635 tablet_service.cc:1514] Processing DeleteTablet for tablet c7f29b6598e6423ebe9b40dc55760a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:17:11 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:48600
I20260806 01:17:11.740983 14202 tablet_service.cc:1514] Processing DeleteTablet for tablet c7f29b6598e6423ebe9b40dc55760a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:17:11 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:54222
I20260806 01:17:11.741725 14431 tablet_service.cc:1514] Processing DeleteTablet for tablet c7f29b6598e6423ebe9b40dc55760a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:17:11 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:37932
I20260806 01:17:11.742453 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 14053
I20260806 01:17:11.754482 14831 tablet_replica.cc:330] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc: stopping tablet replica
I20260806 01:17:11.755676 14830 tablet_replica.cc:330] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e: stopping tablet replica
I20260806 01:17:11.756469 14830 raft_consensus.cc:2229] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:17:11.757200 14828 tablet_replica.cc:330] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b: stopping tablet replica
I20260806 01:17:11.755475 14831 raft_consensus.cc:2229] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:17:11.757881 14831 raft_consensus.cc:2258] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:17:11.758363 14830 raft_consensus.cc:2258] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:17:11.758625 14828 raft_consensus.cc:2229] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:17:11.759378 14828 raft_consensus.cc:2258] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:17:11.761154 14828 ts_tablet_manager.cc:1905] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:17:11.761801 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.744995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54236 (local address 127.25.236.129:46597)
0806 01:17:11.745168 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:11.745174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:11.745678 (+ 504us) server_negotiation.cc:407] Connection header received
0806 01:17:11.745937 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:11.745944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:11.746026 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:11.746102 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:11.748160 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.749042 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.750318 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.750717 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.750853 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.753873 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:11.753886 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:11.753906 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:11.753952 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:11.758294 (+ 4342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:11.759166 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:11.759173 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:11.759176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:11.759345 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:11.759852 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:11.759857 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:11.759860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:11.760288 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:11.760551 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:11.761172 (+ 621us) server_negotiation.cc:299] Negotiation successful
0806 01:17:11.761411 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 01:17:11.761168 14830 ts_tablet_manager.cc:1905] T c7f29b6598e6423ebe9b40dc55760a47 P 882c4701da3941cb981cede16417945e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:17:11.762614 14831 ts_tablet_manager.cc:1905] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:17:11.763160 14287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:11.769129 14828 ts_tablet_manager.cc:1918] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:17:11.769313 14828 log.cc:1196] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/c7f29b6598e6423ebe9b40dc55760a47
I20260806 01:17:11.769881 14828 ts_tablet_manager.cc:1939] T c7f29b6598e6423ebe9b40dc55760a47 P 1d5cf47655bd4844bb376b9c77f61e2b: Deleting consensus metadata
I20260806 01:17:11.772778 13973 catalog_manager.cc:4841] TS 1d5cf47655bd4844bb376b9c77f61e2b (127.25.236.131:35525): tablet c7f29b6598e6423ebe9b40dc55760a47 (table test-table [id=c3b5acdd3c6c4633b7ad580f6bbeb12b]) successfully deleted
I20260806 01:17:11.773906 14831 ts_tablet_manager.cc:1918] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:17:11.774219 14831 log.cc:1196] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/c7f29b6598e6423ebe9b40dc55760a47
I20260806 01:17:11.775405 14831 ts_tablet_manager.cc:1939] T c7f29b6598e6423ebe9b40dc55760a47 P 9f8d2468954b438b996ae64759e431bc: Deleting consensus metadata
I20260806 01:17:11.777993 13974 catalog_manager.cc:4841] TS 9f8d2468954b438b996ae64759e431bc (127.25.236.130:38913): tablet c7f29b6598e6423ebe9b40dc55760a47 (table test-table [id=c3b5acdd3c6c4633b7ad580f6bbeb12b]) successfully deleted
I20260806 01:17:11.899506 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 14053
W20260806 01:17:11.936441 13975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.236.129:46597 (error 108)
I20260806 01:17:11.937145 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 14301
I20260806 01:17:11.942991 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.941652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46597 (local address 127.0.0.1:54248)
0806 01:17:11.942637 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:11.942781 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:46597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":843,"threads_started":1}
W20260806 01:17:11.943410 13975 catalog_manager.cc:4584] TS 882c4701da3941cb981cede16417945e (127.25.236.129:46597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c7f29b6598e6423ebe9b40dc55760a47: Network error: Client connection negotiation failed: client connection to 127.25.236.129:46597: connect: Connection refused (error 111)
I20260806 01:17:11.955914 14799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.937995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37940 (local address 127.25.236.130:38913)
0806 01:17:11.938269 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:11.938279 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:11.938425 (+ 146us) server_negotiation.cc:407] Connection header received
0806 01:17:11.938650 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:11.938658 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:11.938775 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:11.938893 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:11.940901 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.942019 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.944045 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:11.944852 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:11.944989 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:11.948446 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:11.948461 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:11.948481 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:11.948520 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:11.952518 (+ 3998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:11.953461 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:11.953470 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:11.953474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:11.953686 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:11.954092 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:11.954097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:11.954100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:11.954590 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:11.954883 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:11.955304 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 01:17:11.955539 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 01:17:11.956700 14516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:11.997249 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.996683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46597 (local address 127.0.0.1:54256)
0806 01:17:11.996895 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:11.996995 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:46597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 01:17:11.998495 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:11.998053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46597 (local address 127.0.0.1:54264)
0806 01:17:11.998294 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:11.998371 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:46597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 01:17:12.059224 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:12.058782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46597 (local address 127.0.0.1:54266)
0806 01:17:12.058980 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:12.059078 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:46597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 01:17:12.060348 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:12.059985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46597 (local address 127.0.0.1:54268)
0806 01:17:12.060152 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:12.060216 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:46597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 01:17:12.111570 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 14301
I20260806 01:17:12.138474 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 14531
I20260806 01:17:12.140985 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:12.140462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46597 (local address 127.0.0.1:54270)
0806 01:17:12.140684 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:12.140794 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:46597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 01:17:12.142298 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:12.141802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46597 (local address 127.0.0.1:54282)
0806 01:17:12.142028 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:12.142111 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:46597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260806 01:17:12.154364 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:12.139555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48620 (local address 127.25.236.131:35525)
0806 01:17:12.139778 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:12.139784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:12.139805 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:17:12.139973 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:12.139979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:12.140073 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:12.140192 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:12.141874 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:12.142845 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:12.144167 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:12.144695 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:12.144846 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:12.147831 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:12.147843 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:12.147859 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:12.147889 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:12.151248 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:12.151998 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:12.152009 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:12.152011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:12.152159 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:12.152536 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:12.152541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:12.152546 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:12.152917 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:12.153273 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:12.153552 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 01:17:12.153818 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 01:17:12.155079 14731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:12.291522 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 14531
I20260806 01:17:12.315506 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:12.314889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46597 (local address 127.0.0.1:54288)
0806 01:17:12.315151 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:12.315298 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:46597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 01:17:12.316877 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:12.316394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46597 (local address 127.0.0.1:54304)
0806 01:17:12.316589 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:12.316689 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:46597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 01:17:12.329865 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 13944
I20260806 01:17:12.331400 14044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:12.448961 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 13944
I20260806 01:17:12.482857 26546 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 01:17:12.829562 26546 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 01:17:12.829839 26546 mini_ranger.cc:66] Stopping Ranger...
I20260806 01:17:13.186986 26546 mini_ranger.cc:68] Stopped Ranger
2026-08-06 01:17:13.187 UTC [12489] LOG: received smart shutdown request
2026-08-06 01:17:13.190 UTC [12489] LOG: background worker "logical replication launcher" (PID 12505) exited with exit code 1
2026-08-06 01:17:13.191 UTC [12495] LOG: shutting down
2026-08-06 01:17:13.369 UTC [12489] LOG: database system is shut down
2026-08-06T01:17:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56384 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:17:13 dist-test-slave-7nm7 krb5kdc[14851](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:17:13 dist-test-slave-7nm7 krb5kdc[14851](info): set up 2 sockets
Aug 06 01:17:13 dist-test-slave-7nm7 krb5kdc[14851](info): commencing operation
krb5kdc: starting...
W20260806 01:17:14.888586 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:17:15 dist-test-slave-7nm7 krb5kdc[14851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:17:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:17:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:17:15.231123 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:38317
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:46231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:38317,127.25.236.189:39547,127.25.236.188:36587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:15.605495 14866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:15.605808 14866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:15.605901 14866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:15.613335 14866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:17:15.613456 14866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:15.613505 14866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:15.613566 14866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:17:15.613613 14866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:17:15.624390 14866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:46231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:38317,127.25.236.189:39547,127.25.236.188:36587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:38317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:15.625703 14866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:15.626976 14866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:15.640193 14879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:15.640777 14878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:15.640815 14881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:15.641680 14866 server_base.cc:1034] running on GCE node
I20260806 01:17:15.642661 14866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:15.644243 14866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:15.645545 14866 hybrid_clock.cc:648] HybridClock initialized: now 1785979035645466 us; error 69 us; skew 500 ppm
I20260806 01:17:15.645978 14866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:15 dist-test-slave-7nm7 krb5kdc[14851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979035, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:15.650602 14866 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:17:15.652211 14866 webserver.cc:460] Webserver started at http://127.25.236.190:42935/ using document root <none> and password file <none>
I20260806 01:17:15.652971 14866 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:15.653137 14866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:15.653463 14866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:17:15.656705 14866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "d0a4933d42864397858fd74a4fd9b77d"
format_stamp: "Formatted at 2026-08-06 01:17:15 on dist-test-slave-7nm7"
server_key: "39c9ff45111c4ded13dc6dfe3f975d77"
server_key_iv: "f66cf72214252fc7bb66f36a8a9a43f2"
server_key_version: "encryptionkey@0"
I20260806 01:17:15.657559 14866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "d0a4933d42864397858fd74a4fd9b77d"
format_stamp: "Formatted at 2026-08-06 01:17:15 on dist-test-slave-7nm7"
server_key: "39c9ff45111c4ded13dc6dfe3f975d77"
server_key_iv: "f66cf72214252fc7bb66f36a8a9a43f2"
server_key_version: "encryptionkey@0"
I20260806 01:17:15.663017 14866 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 01:17:15.667384 14894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:15.669102 14866 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:17:15.669329 14866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "d0a4933d42864397858fd74a4fd9b77d"
format_stamp: "Formatted at 2026-08-06 01:17:15 on dist-test-slave-7nm7"
server_key: "39c9ff45111c4ded13dc6dfe3f975d77"
server_key_iv: "f66cf72214252fc7bb66f36a8a9a43f2"
server_key_version: "encryptionkey@0"
I20260806 01:17:15.669569 14866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:15.694741 14866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:15.699254 14866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:15.699568 14866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:15.719828 14866 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:38317
I20260806 01:17:15.719847 14964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:38317 every 8 connection(s)
I20260806 01:17:15.721303 14866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:17:15.725503 14965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:15.731176 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 14866
I20260806 01:17:15.731434 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:17:15.732663 26546 external_mini_cluster.cc:1428] Setting key 13e3d56f3b3667c739f647d415bd775d
I20260806 01:17:15.732470 14965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:17:15 dist-test-slave-7nm7 krb5kdc[14851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979035, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:15.759649 14968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:15.736197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58364 (local address 127.25.236.190:38317)
0806 01:17:15.737133 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:15.737150 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:15.737723 (+ 573us) server_negotiation.cc:407] Connection header received
0806 01:17:15.740389 (+ 2666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:15.740399 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:15.740709 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:15.740928 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:15.742464 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:15.744348 (+ 1884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:15.745762 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:15.746635 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:15.746768 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:15.751164 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:15.751172 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:15.751185 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:15.751221 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:15.754934 (+ 3713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:15.755763 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:15.755770 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:15.755772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:15.755917 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:15.756504 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:15.756508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:15.756510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:15.757037 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:15.757258 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:15.757818 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 01:17:15.758134 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":417,"threads_started":1}
I20260806 01:17:15.759684 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:15.736200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:38317 (local address 127.0.0.1:58364)
0806 01:17:15.737534 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:15.737575 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:15.739927 (+ 2352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:15.740984 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:15.740990 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:15.741811 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:15.742184 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:15.742202 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:15.744508 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:15.744512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:15.745604 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:15.745621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:15.746907 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:15.746917 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:15.747157 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:15.747928 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:15.747969 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:15.750910 (+ 2941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:15.755105 (+ 4195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:15.755119 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:15.755132 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:15.755613 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:15.756038 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:15.756044 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:15.756048 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:15.756357 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:15.757184 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:15.757195 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:15.757425 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 01:17:15.757675 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 01:17:15.758311 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"mutex_wait_us":142,"spinlock_wait_cycles":48896,"thread_start_us":842,"threads_started":1}
WARNING: no policy specified for kudu/127.25.236.189@KRBTEST.COM; defaulting to no policy
I20260806 01:17:15.763173 14965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.189@KRBTEST.COM" created.
I20260806 01:17:15.764991 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:15.764197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:39547 (local address 127.0.0.1:53576)
0806 01:17:15.764500 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:15.764719 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260806 01:17:15.765326 14898 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
I20260806 01:17:15.767220 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:15.766678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:39547 (local address 127.0.0.1:53590)
0806 01:17:15.766894 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:15.767005 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 01:17:15.767562 14965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:39547: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.189.keytab.
Entry for principal kudu/127.25.236.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.189.keytab.
I20260806 01:17:15.802157 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.189:39547
--webserver_interface=127.25.236.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.189.keytab
--principal=kudu/127.25.236.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:46231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:38317,127.25.236.189:39547,127.25.236.188:36587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:17:15.817235 14965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } attempt: 1
I20260806 01:17:15.818650 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:15.818036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:39547 (local address 127.0.0.1:53596)
0806 01:17:15.818404 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:15.818524 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
I20260806 01:17:15.819646 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:15.819305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:39547 (local address 127.0.0.1:53608)
0806 01:17:15.819451 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:15.819522 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 01:17:15.819844 14965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:39547: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
I20260806 01:17:15.888136 14965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } attempt: 2
I20260806 01:17:15.889500 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:15.888952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:39547 (local address 127.0.0.1:53616)
0806 01:17:15.889252 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:15.889368 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 01:17:15.890583 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:15.890164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:39547 (local address 127.0.0.1:53630)
0806 01:17:15.890383 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:15.890449 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 01:17:15.890762 14965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:39547: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
I20260806 01:17:15.982017 14965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } attempt: 3
I20260806 01:17:15.983891 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:15.983158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:39547 (local address 127.0.0.1:53640)
0806 01:17:15.983497 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:15.983675 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 01:17:15.985334 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:15.984883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:39547 (local address 127.0.0.1:53656)
0806 01:17:15.985060 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:15.985158 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 01:17:15.985706 14965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:39547: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
I20260806 01:17:16.129006 14965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } attempt: 4
I20260806 01:17:16.130458 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.129781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:39547 (local address 127.0.0.1:53664)
0806 01:17:16.130105 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.130273 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260806 01:17:16.131618 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.131206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:39547 (local address 127.0.0.1:53670)
0806 01:17:16.131374 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.131472 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 01:17:16.131888 14965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:39547: Network error: Client connection negotiation failed: client connection to 127.25.236.189:39547: connect: Connection refused (error 111)
W20260806 01:17:16.168363 14976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:16.168675 14976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:16.168856 14976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:16.176509 14976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:17:16.176637 14976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:16.176738 14976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:16.176828 14976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:17:16.176898 14976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:17:16.188459 14976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:46231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:38317,127.25.236.189:39547,127.25.236.188:36587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.189.keytab
--principal=kudu/127.25.236.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.189:39547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.236.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:16.190044 14976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:16.191478 14976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:16.204084 14987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:16.203984 14986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:16.204164 14991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:16.205540 14976 server_base.cc:1034] running on GCE node
I20260806 01:17:16.206580 14976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:16.208105 14976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:16.209367 14976 hybrid_clock.cc:648] HybridClock initialized: now 1785979036209324 us; error 37 us; skew 500 ppm
I20260806 01:17:16.209774 14976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:16 dist-test-slave-7nm7 krb5kdc[14851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979036, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:16.213649 14976 init.cc:377] Logged in from keytab as kudu/127.25.236.189@KRBTEST.COM (short username kudu)
I20260806 01:17:16.215548 14976 webserver.cc:460] Webserver started at http://127.25.236.189:44285/ using document root <none> and password file <none>
I20260806 01:17:16.216262 14976 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:16.216372 14976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:16.216632 14976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:17:16.220021 14976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/data/instance:
uuid: "c33e540a9fa9461698644d7c8fb8d352"
format_stamp: "Formatted at 2026-08-06 01:17:16 on dist-test-slave-7nm7"
server_key: "79164c5148252f08800036d4c784cbb2"
server_key_iv: "93bc5b5826f4b63346dc14362a790ef7"
server_key_version: "encryptionkey@0"
I20260806 01:17:16.221005 14976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/wal/instance:
uuid: "c33e540a9fa9461698644d7c8fb8d352"
format_stamp: "Formatted at 2026-08-06 01:17:16 on dist-test-slave-7nm7"
server_key: "79164c5148252f08800036d4c784cbb2"
server_key_iv: "93bc5b5826f4b63346dc14362a790ef7"
server_key_version: "encryptionkey@0"
I20260806 01:17:16.227329 14976 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.007s
I20260806 01:17:16.231408 15000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:16.233456 14976 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260806 01:17:16.233673 14976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/wal
uuid: "c33e540a9fa9461698644d7c8fb8d352"
format_stamp: "Formatted at 2026-08-06 01:17:16 on dist-test-slave-7nm7"
server_key: "79164c5148252f08800036d4c784cbb2"
server_key_iv: "93bc5b5826f4b63346dc14362a790ef7"
server_key_version: "encryptionkey@0"
I20260806 01:17:16.233902 14976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:16.248522 14976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:16.253453 14976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:16.253839 14976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:16.275660 14976 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.189:39547
I20260806 01:17:16.275701 15071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.189:39547 every 8 connection(s)
I20260806 01:17:16.277123 14976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/data/info.pb
I20260806 01:17:16.281669 15073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:16.284683 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 14976
I20260806 01:17:16.284943 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-1/wal/instance
I20260806 01:17:16.285456 26546 external_mini_cluster.cc:1428] Setting key 533c667b620f0522aa2a1cfeedaee198
I20260806 01:17:16.288796 15073 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:17:16 dist-test-slave-7nm7 krb5kdc[14851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979036, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.189@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:16.316336 15077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.291728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58374 (local address 127.25.236.190:38317)
0806 01:17:16.292303 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:16.292308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:16.293532 (+ 1224us) server_negotiation.cc:407] Connection header received
0806 01:17:16.295748 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:16.295755 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:16.295858 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:16.295980 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:16.297611 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.298486 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.301031 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.301500 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.301603 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.306616 (+ 5013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:16.306625 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:16.306633 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:16.306697 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:16.311820 (+ 5123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.312794 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.312803 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.312810 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.312978 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.313567 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.313576 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.313582 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.314059 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:16.314437 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:16.315597 (+ 1160us) server_negotiation.cc:299] Negotiation successful
0806 01:17:16.315870 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":347,"threads_started":1}
I20260806 01:17:16.317687 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.292321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:38317 (local address 127.0.0.1:58374)
0806 01:17:16.293333 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.293387 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:16.295442 (+ 2055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:16.296235 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:16.296243 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:16.296788 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:16.297360 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.297376 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.298668 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.298674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.300858 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.300876 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.301766 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.301777 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.302091 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.302933 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:16.302971 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:16.306306 (+ 3335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:16.312065 (+ 5759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.312075 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.312089 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.312627 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.313129 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.313132 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.313135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.313403 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.314410 (+ 1007us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:16.314436 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:16.315102 (+ 666us) client_negotiation.cc:769] Sending connection context
0806 01:17:16.315437 (+ 335us) client_negotiation.cc:240] Negotiation successful
0806 01:17:16.315969 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":378,"threads_started":1}
WARNING: no policy specified for kudu/127.25.236.188@KRBTEST.COM; defaulting to no policy
I20260806 01:17:16.319712 15073 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.188@KRBTEST.COM" created.
Aug 06 01:17:16 dist-test-slave-7nm7 krb5kdc[14851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979036, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.189@KRBTEST.COM for kudu/127.25.236.189@KRBTEST.COM
I20260806 01:17:16.339967 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.320988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53680 (local address 127.25.236.189:39547)
0806 01:17:16.321796 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:16.321802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:16.321828 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:17:16.321875 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:16.321881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:16.322057 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:16.322210 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:16.323520 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.324919 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.326807 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.327670 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.327978 (+ 308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.332333 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:16.332352 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:16.332375 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:16.332420 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:16.336469 (+ 4049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.337471 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.337479 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.337481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.337610 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.338106 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.338110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.338112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.338571 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:16.338740 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:16.339435 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 01:17:16.339642 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":484,"threads_started":1}
I20260806 01:17:16.340004 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.320657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:39547 (local address 127.0.0.1:53680)
0806 01:17:16.320860 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.320884 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:16.321002 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:16.322271 (+ 1269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:16.322279 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:16.322835 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:16.323064 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.323182 (+ 118us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.325084 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.325092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.326646 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.326665 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.327996 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.328005 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.328338 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.329054 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:16.329082 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:16.331991 (+ 2909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:16.336643 (+ 4652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.336652 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.336659 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.337296 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.337733 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.337741 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.337747 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.337968 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.338713 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:16.338721 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:16.338891 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 01:17:16.339044 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 01:17:16.339393 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 01:17:16.342756 15073 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:17:16.343941 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.343466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57050)
0806 01:17:16.343665 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.343786 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 01:17:16.344247 15005 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
I20260806 01:17:16.346565 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.346059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57054)
0806 01:17:16.346347 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.346440 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260806 01:17:16.346854 15073 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:36587: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.188.keytab.
Entry for principal kudu/127.25.236.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.188.keytab.
I20260806 01:17:16.362552 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.188:36587
--webserver_interface=127.25.236.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.188.keytab
--principal=kudu/127.25.236.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:46231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:38317,127.25.236.189:39547,127.25.236.188:36587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:17:16.396237 15073 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } attempt: 1
I20260806 01:17:16.397660 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.397023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57062)
0806 01:17:16.397294 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.397440 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 01:17:16.398798 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.398438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57064)
0806 01:17:16.398616 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.398693 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 01:17:16.398988 15073 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:36587: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
I20260806 01:17:16.431191 14965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } attempt: 5
Aug 06 01:17:16 dist-test-slave-7nm7 krb5kdc[14851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979035, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for kudu/127.25.236.189@KRBTEST.COM
I20260806 01:17:16.449450 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.432002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:39547 (local address 127.0.0.1:53688)
0806 01:17:16.432235 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.432259 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:16.432393 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:16.432792 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:16.432800 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:16.433220 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:16.433390 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.433399 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.434744 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.434750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.436243 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.436262 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.437207 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.437213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.437530 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.438212 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:16.438287 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:16.441531 (+ 3244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:16.446401 (+ 4870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.446409 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.446413 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.447044 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.447546 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.447553 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.447558 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.447743 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.448410 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:16.448415 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:16.448585 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 01:17:16.448758 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 01:17:16.449109 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 01:17:16.449450 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.432138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53688 (local address 127.25.236.189:39547)
0806 01:17:16.432348 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:16.432353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:16.432374 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:17:16.432546 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:16.432552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:16.432637 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:16.432749 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:16.433556 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.434485 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.436519 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.437009 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.437178 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.441799 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:16.441806 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:16.441811 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:16.441860 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:16.446120 (+ 4260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.447227 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.447234 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.447237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.447392 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.447889 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.447894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.447897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.448278 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:16.448445 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:16.448863 (+ 418us) server_negotiation.cc:299] Negotiation successful
0806 01:17:16.449092 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 01:17:16.450987 14965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:17:16.452242 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.451719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57072)
0806 01:17:16.451978 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.452083 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260806 01:17:16.459055 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.452861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57084)
0806 01:17:16.457950 (+ 5089us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.458057 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4944,"threads_started":1}
W20260806 01:17:16.459580 14965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:36587: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
I20260806 01:17:16.467202 15073 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } attempt: 2
I20260806 01:17:16.468377 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.467854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57096)
0806 01:17:16.468076 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.468188 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 01:17:16.469455 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.469067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57108)
0806 01:17:16.469218 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.469302 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 01:17:16.469677 15073 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:36587: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
I20260806 01:17:16.510824 14965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } attempt: 1
I20260806 01:17:16.512274 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.511637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57122)
0806 01:17:16.511875 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.512044 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 01:17:16.513454 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.513081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57138)
0806 01:17:16.513267 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.513340 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260806 01:17:16.513660 14965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:36587: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
I20260806 01:17:16.560921 15073 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } attempt: 3
I20260806 01:17:16.562320 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.561747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57148)
0806 01:17:16.561974 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.562095 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 01:17:16.563498 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.563004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57162)
0806 01:17:16.563262 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.563356 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260806 01:17:16.563786 15073 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:36587: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
I20260806 01:17:16.581911 14965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } attempt: 2
I20260806 01:17:16.583536 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.582767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57168)
0806 01:17:16.583160 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.583307 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260806 01:17:16.585140 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.584621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57172)
0806 01:17:16.584871 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.584963 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260806 01:17:16.585448 14965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:36587: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
I20260806 01:17:16.691704 14965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } attempt: 3
I20260806 01:17:16.692883 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.692438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57186)
0806 01:17:16.692650 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.692736 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 01:17:16.694193 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.693838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57188)
0806 01:17:16.694000 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.694080 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 01:17:16.694438 14965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:36587: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
I20260806 01:17:16.707026 15073 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } attempt: 4
I20260806 01:17:16.708369 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.707742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57192)
0806 01:17:16.708023 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.708142 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 01:17:16.709570 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.709134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57196)
0806 01:17:16.709322 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.709421 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 01:17:16.709825 15073 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:36587: Network error: Client connection negotiation failed: client connection to 127.25.236.188:36587: connect: Connection refused (error 111)
W20260806 01:17:16.741731 15085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:16.742007 15085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:16.742178 15085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:16.749682 15085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:17:16.749799 15085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:16.749846 15085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:16.749903 15085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:17:16.749948 15085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:17:16.760213 15085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:46231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:38317,127.25.236.189:39547,127.25.236.188:36587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.188.keytab
--principal=kudu/127.25.236.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.188:36587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.236.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:16.761585 15085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:16.762949 15085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:16.777261 15094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:16.777391 15099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:16.777344 15095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:16.778707 15085 server_base.cc:1034] running on GCE node
I20260806 01:17:16.779695 15085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:16.781201 15085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:16.782382 15085 hybrid_clock.cc:648] HybridClock initialized: now 1785979036782307 us; error 58 us; skew 500 ppm
I20260806 01:17:16.782697 15085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:16 dist-test-slave-7nm7 krb5kdc[14851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979036, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:16.786010 15085 init.cc:377] Logged in from keytab as kudu/127.25.236.188@KRBTEST.COM (short username kudu)
I20260806 01:17:16.787936 15085 webserver.cc:460] Webserver started at http://127.25.236.188:32829/ using document root <none> and password file <none>
I20260806 01:17:16.788666 15085 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:16.788810 15085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:16.789139 15085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:17:16.793197 15085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/data/instance:
uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3"
format_stamp: "Formatted at 2026-08-06 01:17:16 on dist-test-slave-7nm7"
server_key: "ebfd32783faa5eeb3a480dda526bbad0"
server_key_iv: "7905337de25f7011355719d985d7e5bc"
server_key_version: "encryptionkey@0"
I20260806 01:17:16.794562 15085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/wal/instance:
uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3"
format_stamp: "Formatted at 2026-08-06 01:17:16 on dist-test-slave-7nm7"
server_key: "ebfd32783faa5eeb3a480dda526bbad0"
server_key_iv: "7905337de25f7011355719d985d7e5bc"
server_key_version: "encryptionkey@0"
I20260806 01:17:16.800396 15085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:17:16.804118 15107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:16.805755 15085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:17:16.805948 15085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/wal
uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3"
format_stamp: "Formatted at 2026-08-06 01:17:16 on dist-test-slave-7nm7"
server_key: "ebfd32783faa5eeb3a480dda526bbad0"
server_key_iv: "7905337de25f7011355719d985d7e5bc"
server_key_version: "encryptionkey@0"
I20260806 01:17:16.806155 15085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:16.822867 15085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:16.828783 15085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:16.829252 15085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:16.852963 15085 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.188:36587
I20260806 01:17:16.853091 15183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.188:36587 every 8 connection(s)
I20260806 01:17:16.854517 15085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/data/info.pb
I20260806 01:17:16.855106 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 15085
I20260806 01:17:16.855321 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978805316623-26546-0/minicluster-data/master-2/wal/instance
I20260806 01:17:16.856009 26546 external_mini_cluster.cc:1428] Setting key c1d71852158074c1106227f0784190fa
I20260806 01:17:16.859093 15187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:16.869916 15187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:17:16.871686 14965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } attempt: 4
Aug 06 01:17:16 dist-test-slave-7nm7 krb5kdc[14851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.188@KRBTEST.COM
Aug 06 01:17:16 dist-test-slave-7nm7 krb5kdc[14851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979035, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for kudu/127.25.236.188@KRBTEST.COM
I20260806 01:17:16.886977 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.858858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57212 (local address 127.25.236.188:36587)
0806 01:17:16.859804 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:16.859816 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:16.859861 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:17:16.862679 (+ 2818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:16.862751 (+ 72us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:16.863611 (+ 860us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:16.864455 (+ 844us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:16.866209 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.867631 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.869071 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.869744 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.869941 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.875514 (+ 5573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:16.875527 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:16.875550 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:16.875612 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:16.879879 (+ 4267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.881104 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.881112 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.881261 (+ 149us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.881526 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.882176 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.882183 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.882188 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.882990 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:16.883296 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:16.884314 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0806 01:17:16.884799 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":444,"threads_started":1}
Aug 06 01:17:16 dist-test-slave-7nm7 krb5kdc[14851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979036, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.188@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:16.896366 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.872606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57220 (local address 127.25.236.188:36587)
0806 01:17:16.873403 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:16.873409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:16.873431 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:17:16.873481 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:16.873486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:16.873561 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:16.875189 (+ 1628us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:16.875498 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.877877 (+ 2379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.880575 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.881370 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.881495 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.886852 (+ 5357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:16.886860 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:16.886864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:16.886904 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:16.892066 (+ 5162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.893278 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.893288 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.893293 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.893680 (+ 387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.894263 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.894269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.894272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.894850 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:16.895008 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:16.895595 (+ 587us) server_negotiation.cc:299] Negotiation successful
0806 01:17:16.895812 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":469,"threads_started":1}
I20260806 01:17:16.896808 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.872388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57220)
0806 01:17:16.872671 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.872692 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:16.872820 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:16.873822 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:16.873827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:16.874688 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:16.874951 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.874966 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.878101 (+ 3135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.878236 (+ 135us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.880240 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.880282 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.881572 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.881582 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.881986 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.882747 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:16.882765 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:16.886608 (+ 3843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:16.892260 (+ 5652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.892266 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.892269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.892888 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.893860 (+ 972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.893867 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.893871 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.894080 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.895084 (+ 1004us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:16.895089 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:16.895275 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 01:17:16.895493 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 01:17:16.895829 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260806 01:17:16.900437 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.873652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:38317 (local address 127.0.0.1:58382)
0806 01:17:16.877712 (+ 4060us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.877764 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:16.877957 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:16.878395 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:16.878405 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:16.880436 (+ 2031us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:16.880825 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.880836 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.882506 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.882512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.883860 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.883872 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.885643 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.885652 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.886027 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.887149 (+ 1122us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:16.887168 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:16.890966 (+ 3798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:16.897263 (+ 6297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.897278 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.897286 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.897917 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.898365 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.898371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.898374 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.898522 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.899184 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:16.899195 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:16.899392 (+ 197us) client_negotiation.cc:769] Sending connection context
0806 01:17:16.899610 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 01:17:16.899951 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3591,"mutex_wait_us":1,"thread_start_us":3280,"threads_started":1}
I20260806 01:17:16.900701 15193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.873660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58382 (local address 127.25.236.190:38317)
0806 01:17:16.877958 (+ 4298us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:16.877965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:16.877985 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:17:16.878124 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:16.878130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:16.878206 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:16.878355 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:16.881092 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.882305 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.884015 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.884919 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.885040 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.891232 (+ 6192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:16.891240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:16.891244 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:16.891490 (+ 246us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:16.897091 (+ 5601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.898072 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.898078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.898088 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.898257 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.898656 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.898661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.898665 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.899018 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:16.899188 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:16.899760 (+ 572us) server_negotiation.cc:299] Negotiation successful
0806 01:17:16.900001 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4182,"thread_start_us":2995,"threads_started":1}
I20260806 01:17:16.903219 14965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d: Bootstrap starting.
Aug 06 01:17:16 dist-test-slave-7nm7 krb5kdc[14851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:16.907433 14965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:16.908782 15187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:17:16.909703 14965 log.cc:824] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:16.914332 15195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.891702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58386 (local address 127.25.236.190:38317)
0806 01:17:16.892584 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:16.892591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:16.892616 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:17:16.892673 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:16.892679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:16.892761 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:16.894364 (+ 1603us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:16.895513 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.897139 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.899015 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.899536 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.900517 (+ 981us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.905707 (+ 5190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:16.905712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:16.905716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:16.905748 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:16.910470 (+ 4722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.911589 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.911600 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.911606 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.912001 (+ 395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.912504 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.912510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.912514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.912928 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:16.913214 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:16.913533 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 01:17:16.913776 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":481,"threads_started":1}
I20260806 01:17:16.914443 14965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d: No bootstrap required, opened a new log
Aug 06 01:17:16 dist-test-slave-7nm7 krb5kdc[14851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979036, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.188@KRBTEST.COM for kudu/127.25.236.189@KRBTEST.COM
I20260806 01:17:16.924016 14965 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } }
I20260806 01:17:16.924458 14965 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:16.924629 14965 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0a4933d42864397858fd74a4fd9b77d, State: Initialized, Role: FOLLOWER
I20260806 01:17:16.925429 14965 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } }
I20260806 01:17:16.926841 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.909764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:39547 (local address 127.0.0.1:53702)
0806 01:17:16.910073 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.910096 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:16.910276 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:16.910933 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:16.910939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:16.911458 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:16.911633 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.911642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.912898 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.912904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.914508 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.914524 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.915553 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.915559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.915754 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.916412 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:16.916437 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:16.919778 (+ 3341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:16.923988 (+ 4210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.923996 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.924002 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.924477 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.924929 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.924934 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.924938 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.925149 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.925740 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:16.925745 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:16.925901 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 01:17:16.926043 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 01:17:16.926464 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260806 01:17:16.926875 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.910176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53702 (local address 127.25.236.189:39547)
0806 01:17:16.910403 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:16.910410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:16.910451 (+ 41us) server_negotiation.cc:407] Connection header received
0806 01:17:16.910517 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:16.910525 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:16.910612 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:16.910719 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:16.911780 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.912708 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.914710 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.915397 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.915534 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.920074 (+ 4540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:16.920080 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:16.920084 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:16.920122 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:16.923740 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.924622 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.924628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.924631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.924823 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.925327 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.925332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.925336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.925607 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:16.925770 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:16.926276 (+ 506us) server_negotiation.cc:299] Negotiation successful
0806 01:17:16.926495 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 01:17:16.927842 15204 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } } }
I20260806 01:17:16.928373 15204 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:17:16.928455 15187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:17:16 dist-test-slave-7nm7 krb5kdc[14851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.189@KRBTEST.COM
I20260806 01:17:16.930153 14965 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 01:17:16 dist-test-slave-7nm7 krb5kdc[14851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979036, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.188@KRBTEST.COM for kudu/127.25.236.188@KRBTEST.COM
I20260806 01:17:16.939393 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.916516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53710 (local address 127.25.236.189:39547)
0806 01:17:16.917197 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:16.917205 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:16.917571 (+ 366us) server_negotiation.cc:407] Connection header received
0806 01:17:16.917804 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:16.917809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:16.917884 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:16.917997 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:16.919909 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.921221 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.923258 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.923926 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.924060 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.931189 (+ 7129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:16.931196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:16.931201 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:16.931236 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:16.935687 (+ 4451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.936744 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.936751 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.936755 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.936915 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.937304 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.937309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.937313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.937687 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:16.938042 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:16.938644 (+ 602us) server_negotiation.cc:299] Negotiation successful
0806 01:17:16.938965 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":370,"threads_started":1}
I20260806 01:17:16.944307 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:17:16.946269 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.929185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57234)
0806 01:17:16.929368 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:16.929387 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:16.929478 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:16.929797 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:16.929801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:16.930198 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:16.930429 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.930437 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.931585 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.931590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.932982 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:16.932998 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.934143 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.934149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:16.934505 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.935299 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:16.935337 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:16.939547 (+ 4210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:16.943518 (+ 3971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.943523 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.943526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.944035 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.944452 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:16.944456 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:16.944459 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:16.944607 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:16.945284 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:16.945288 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:16.945438 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 01:17:16.945584 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 01:17:16.945886 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 01:17:16.947113 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.929222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57234 (local address 127.25.236.188:36587)
0806 01:17:16.929399 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:16.929406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:16.929438 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:17:16.929602 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:16.929607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:16.929685 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:16.929782 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:16.930580 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.931429 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.933141 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.933810 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.933913 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.939766 (+ 5853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:16.939773 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:16.939780 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:16.939825 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:16.943354 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.944167 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.944172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.944177 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.944327 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.944711 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.944715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.944718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.945095 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:16.945240 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:16.946587 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0806 01:17:16.946806 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 01:17:16.953052 15187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3: Bootstrap starting.
I20260806 01:17:16.942160 14866 master_runner.cc:186] Error getting master registration for 127.25.236.189:39547: OK, instance_id { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" instance_seqno: 1785979036269291 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:17:16.955288 14866 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:17:16.956854 15187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:16.957934 15187 log.cc:824] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3: Log is configured to *not* fsync() on all Append() calls
W20260806 01:17:16.958942 14901 tablet.cc:2369] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:17:16.961680 15220 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:17:16.961812 15220 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:17:16.965673 15187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3: No bootstrap required, opened a new log
I20260806 01:17:16.980595 15187 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } }
I20260806 01:17:16.984826 15195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.961067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.25.236.190:38317)
0806 01:17:16.961285 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:16.961294 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:16.963784 (+ 2490us) server_negotiation.cc:407] Connection header received
0806 01:17:16.964057 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:16.964064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:16.964160 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:16.964245 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:16.967545 (+ 3300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.968752 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.970813 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.971762 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.971942 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.976842 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:16.976849 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:16.976854 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:16.976902 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:16.980973 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.982103 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.982110 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.982114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.982308 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.982846 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.982852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.982861 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.983246 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:16.983520 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:16.984177 (+ 657us) server_negotiation.cc:299] Negotiation successful
0806 01:17:16.984439 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 01:17:16.985368 15187 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:16.985613 15187 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3ce8a5b6acf46f3a2a9dc29db4c77f3, State: Initialized, Role: FOLLOWER
I20260806 01:17:16.989821 15187 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } }
I20260806 01:17:16.992448 15234 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } } }
I20260806 01:17:16.992971 15234 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:17:16.996059 15187 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:17:16.998844 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.964144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53724 (local address 127.25.236.189:39547)
0806 01:17:16.964390 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:16.964397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:16.972127 (+ 7730us) server_negotiation.cc:407] Connection header received
0806 01:17:16.973513 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:16.973519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:16.973605 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:16.973676 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:16.979350 (+ 5674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.980664 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.983988 (+ 3324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.984761 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.984903 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.989497 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:16.989504 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:16.989509 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:16.989556 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:16.993937 (+ 4381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.995550 (+ 1613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.995558 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.995561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.995748 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:16.996329 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:16.996335 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:16.996339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:16.996720 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:16.996984 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:16.998157 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0806 01:17:16.998455 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 01:17:17.005959 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:16.972671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57244 (local address 127.25.236.188:36587)
0806 01:17:16.972866 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:16.972873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:16.982711 (+ 9838us) server_negotiation.cc:407] Connection header received
0806 01:17:16.983006 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:16.983013 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:16.983094 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:16.983165 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:16.988438 (+ 5273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.989516 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.992042 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:16.992810 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:16.992914 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:16.996925 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:16.996933 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:16.996938 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:16.996994 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:17.001955 (+ 4961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:17.003167 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:17.003179 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:17.003185 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:17.003421 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:17.004020 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:17.004025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:17.004029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:17.004394 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:17.004729 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:17.005227 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 01:17:17.005486 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:17:17.009091 15073 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } attempt: 5
Aug 06 01:17:17 dist-test-slave-7nm7 krb5kdc[14851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979036, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.189@KRBTEST.COM for kudu/127.25.236.188@KRBTEST.COM
I20260806 01:17:17.029837 15078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:17.009766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:36587 (local address 127.0.0.1:57258)
0806 01:17:17.009942 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:17.009964 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:17.010082 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:17.010617 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:17.010622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:17.011189 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:17.011401 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:17.011413 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:17.013472 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:17.013480 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:17.015462 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:17.015480 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:17.016802 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:17.016809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:17.017033 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:17.017852 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:17.017877 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:17.020899 (+ 3022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:17.026125 (+ 5226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:17.026131 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:17.026135 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:17.026717 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:17.027337 (+ 620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:17.027343 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:17.027347 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:17.027550 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:17.028703 (+ 1153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:17.028708 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:17.028884 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 01:17:17.029041 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 01:17:17.029396 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 01:17:17.031509 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:17.010000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57258 (local address 127.25.236.188:36587)
0806 01:17:17.010169 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:17.010176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:17.010193 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:17:17.010270 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:17.010275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:17.010346 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:17.010436 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:17.011623 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:17.013067 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:17.015676 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:17.016428 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:17.016565 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:17.021187 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:17.021195 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:17.021202 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:17.021251 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:17.025913 (+ 4662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:17.026874 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:17.026880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:17.026887 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:17.027039 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:17.027716 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:17.027721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:17.027725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:17.028405 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:17.028613 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:17.030747 (+ 2134us) server_negotiation.cc:299] Negotiation successful
0806 01:17:17.031015 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 01:17:17.038883 15073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352: Bootstrap starting.
I20260806 01:17:17.043582 15073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:17.045130 15073 log.cc:824] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:17.047322 15234 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:17:17.047677 15234 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } }
I20260806 01:17:17.049374 15234 leader_election.cc:288] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0a4933d42864397858fd74a4fd9b77d (127.25.236.190:38317), c33e540a9fa9461698644d7c8fb8d352 (127.25.236.189:39547)
I20260806 01:17:17.051592 14930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0a4933d42864397858fd74a4fd9b77d" is_pre_election: true
I20260806 01:17:17.052255 14930 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3ce8a5b6acf46f3a2a9dc29db4c77f3 in term 0.
I20260806 01:17:17.053453 15113 leader_election.cc:302] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3ce8a5b6acf46f3a2a9dc29db4c77f3, d0a4933d42864397858fd74a4fd9b77d; no voters:
I20260806 01:17:17.053970 15234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:17:17.054139 15234 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:17:17.054296 15234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:17.062026 15234 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } }
I20260806 01:17:17.063040 15234 leader_election.cc:288] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [CANDIDATE]: Term 1 election: Requested vote from peers d0a4933d42864397858fd74a4fd9b77d (127.25.236.190:38317), c33e540a9fa9461698644d7c8fb8d352 (127.25.236.189:39547)
I20260806 01:17:17.030419 15085 master_runner.cc:186] Error getting master registration for 127.25.236.189:39547: OK, instance_id { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" instance_seqno: 1785979036269291 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:17:17.065864 15085 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:17:17.066104 14930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0a4933d42864397858fd74a4fd9b77d"
I20260806 01:17:17.066370 14930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:17.069118 15073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352: No bootstrap required, opened a new log
I20260806 01:17:17.069890 14930 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3ce8a5b6acf46f3a2a9dc29db4c77f3 in term 1.
I20260806 01:17:17.070607 15113 leader_election.cc:302] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3ce8a5b6acf46f3a2a9dc29db4c77f3, d0a4933d42864397858fd74a4fd9b77d; no voters:
I20260806 01:17:17.070909 15234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [term 1 FOLLOWER]: Leader election won for term 1
W20260806 01:17:17.077072 15251 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:17:17.077291 15251 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 01:17:17.082304 15115 tablet.cc:2369] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:17:17.085328 15073 raft_consensus.cc:348] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } }
I20260806 01:17:17.085760 15073 raft_consensus.cc:374] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:17.085887 15073 raft_consensus.cc:729] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c33e540a9fa9461698644d7c8fb8d352, State: Initialized, Role: FOLLOWER
I20260806 01:17:17.086659 15073 consensus_queue.cc:260] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } }
I20260806 01:17:17.091264 15234 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [term 1 LEADER]: Becoming Leader. State: Replica: c3ce8a5b6acf46f3a2a9dc29db4c77f3, State: Running, Role: LEADER
I20260806 01:17:17.091843 15234 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } }
I20260806 01:17:17.093632 15257 sys_catalog.cc:455] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } } }
I20260806 01:17:17.095938 15257 sys_catalog.cc:458] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:17:17.095872 15034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c33e540a9fa9461698644d7c8fb8d352"
I20260806 01:17:17.096413 15034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:17.096377 15256 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3ce8a5b6acf46f3a2a9dc29db4c77f3. Latest consensus state: current_term: 1 leader_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } } }
I20260806 01:17:17.096812 15256 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:17.100593 15034 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3ce8a5b6acf46f3a2a9dc29db4c77f3 in term 1.
I20260806 01:17:17.095928 15035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c33e540a9fa9461698644d7c8fb8d352" is_pre_election: true
I20260806 01:17:17.101738 15035 raft_consensus.cc:2362] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c3ce8a5b6acf46f3a2a9dc29db4c77f3 in term 1. Re-sending same reply.
I20260806 01:17:17.095865 15073 sys_catalog.cc:564] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:17:17.106618 15261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:17:17.109014 15261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 01:17:17.137068 15274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:17:17.137377 15274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:17:17.148538 14930 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [term 1 FOLLOWER]: Refusing update from remote peer c3ce8a5b6acf46f3a2a9dc29db4c77f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:17:17.149786 15034 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [term 1 FOLLOWER]: Refusing update from remote peer c3ce8a5b6acf46f3a2a9dc29db4c77f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:17:17.151247 15234 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:17:17.151589 15256 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:17:17.180186 15257 sys_catalog.cc:455] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3ce8a5b6acf46f3a2a9dc29db4c77f3. Latest consensus state: current_term: 1 leader_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } } }
I20260806 01:17:17.180513 15257 sys_catalog.cc:458] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:17:17.196475 15256 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } } }
I20260806 01:17:17.196801 15256 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:17.199293 15261 catalog_manager.cc:1347] Generated new cluster ID: b01d29aafe4546fab016cff7626c1129
I20260806 01:17:17.199420 15261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:17:17.199038 15204 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3ce8a5b6acf46f3a2a9dc29db4c77f3. Latest consensus state: current_term: 1 leader_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } } }
I20260806 01:17:17.199535 15204 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:17:17.211055 15234 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } } }
I20260806 01:17:17.211386 15234 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:17.229080 15204 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } } }
I20260806 01:17:17.229377 15204 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:17:17.229962 15257 sys_catalog.cc:455] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0a4933d42864397858fd74a4fd9b77d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38317 } } peers { permanent_uuid: "c33e540a9fa9461698644d7c8fb8d352" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 39547 } } peers { permanent_uuid: "c3ce8a5b6acf46f3a2a9dc29db4c77f3" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 36587 } } }
I20260806 01:17:17.231942 15257 sys_catalog.cc:458] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:17:17.254451 15261 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:17:17.256814 15261 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:17:17.295488 15261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3: Generated new TSK 0
I20260806 01:17:17.296913 15261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:17:17.964138 15220 catalog_manager.cc:1259] Loaded cluster ID: b01d29aafe4546fab016cff7626c1129
I20260806 01:17:17.964257 15220 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d: loading cluster ID for follower catalog manager: success
I20260806 01:17:17.967016 15220 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d: acquiring CA information for follower catalog manager: success
I20260806 01:17:17.968988 15220 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d0a4933d42864397858fd74a4fd9b77d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:17:18.139991 15274 catalog_manager.cc:1259] Loaded cluster ID: b01d29aafe4546fab016cff7626c1129
I20260806 01:17:18.140156 15274 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352: loading cluster ID for follower catalog manager: success
I20260806 01:17:18.142733 15274 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352: acquiring CA information for follower catalog manager: success
I20260806 01:17:18.144549 15274 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c33e540a9fa9461698644d7c8fb8d352: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:17:18.445250 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 14866
I20260806 01:17:18.446976 14957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:18.571533 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 14866
I20260806 01:17:18.604876 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 14976
I20260806 01:17:18.606513 15067 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:18.709434 15294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:18.708562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:38317 (local address 127.0.0.1:58408)
0806 01:17:18.709071 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:18.709268 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:38317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":301,"threads_started":1}
W20260806 01:17:18.709805 15113 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.236.190:38317: connect: Connection refused (error 111)
I20260806 01:17:18.711704 15294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:18.710954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:38317 (local address 127.0.0.1:58414)
0806 01:17:18.711344 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:18.711513 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:38317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260806 01:17:18.712096 15113 consensus_peers.cc:487] T 00000000000000000000000000000000 P c3ce8a5b6acf46f3a2a9dc29db4c77f3 -> Peer d0a4933d42864397858fd74a4fd9b77d (127.25.236.190:38317): Couldn't send request to peer d0a4933d42864397858fd74a4fd9b77d. Status: Network error: Client connection negotiation failed: client connection to 127.25.236.190:38317: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 01:17:18.721941 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 14976
I20260806 01:17:18.748752 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 15085
I20260806 01:17:18.750054 15178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:18.865430 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 15085
2026-08-06T01:17:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5360 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:17:18 dist-test-slave-7nm7 krb5kdc[15303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:17:18 dist-test-slave-7nm7 krb5kdc[15303](info): set up 2 sockets
Aug 06 01:17:18 dist-test-slave-7nm7 krb5kdc[15303](info): commencing operation
krb5kdc: starting...
W20260806 01:17:20.235488 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:17:20 dist-test-slave-7nm7 krb5kdc[15303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:17:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:17:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:17:20.579218 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:44931
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:44601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:44931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:20.964425 15320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:20.964823 15320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:20.964996 15320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:20.973641 15320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:17:20.973773 15320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:20.973876 15320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:20.973973 15320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:17:20.974068 15320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:17:20.985108 15320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:44601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:44931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:44931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:20.987049 15320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:20.988418 15320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:21.001389 15334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:21.001475 15332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:21.001726 15330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:21.002497 15320 server_base.cc:1034] running on GCE node
I20260806 01:17:21.003441 15320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:21.004907 15320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:21.006150 15320 hybrid_clock.cc:648] HybridClock initialized: now 1785979041006093 us; error 33 us; skew 500 ppm
I20260806 01:17:21.006600 15320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:21 dist-test-slave-7nm7 krb5kdc[15303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979041, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:21.009886 15320 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:17:21.011847 15320 webserver.cc:460] Webserver started at http://127.25.236.190:41735/ using document root <none> and password file <none>
I20260806 01:17:21.012560 15320 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:21.012696 15320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:21.013047 15320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:17:21.016263 15320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "3bd865854b78493e8c3d52ef1e373a35"
format_stamp: "Formatted at 2026-08-06 01:17:21 on dist-test-slave-7nm7"
server_key: "ffbf20aa2a0c312011555690159e0fda"
server_key_iv: "b548d09ac90d18f988c5fcf296f57ec7"
server_key_version: "encryptionkey@0"
I20260806 01:17:21.017035 15320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "3bd865854b78493e8c3d52ef1e373a35"
format_stamp: "Formatted at 2026-08-06 01:17:21 on dist-test-slave-7nm7"
server_key: "ffbf20aa2a0c312011555690159e0fda"
server_key_iv: "b548d09ac90d18f988c5fcf296f57ec7"
server_key_version: "encryptionkey@0"
I20260806 01:17:21.022773 15320 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:17:21.026648 15349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:21.028134 15320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:17:21.028358 15320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "3bd865854b78493e8c3d52ef1e373a35"
format_stamp: "Formatted at 2026-08-06 01:17:21 on dist-test-slave-7nm7"
server_key: "ffbf20aa2a0c312011555690159e0fda"
server_key_iv: "b548d09ac90d18f988c5fcf296f57ec7"
server_key_version: "encryptionkey@0"
I20260806 01:17:21.028600 15320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:21.045974 15320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:21.050514 15320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:21.050838 15320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:21.071657 15320 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:44931
I20260806 01:17:21.071666 15432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:44931 every 8 connection(s)
I20260806 01:17:21.073516 15320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:17:21.077694 15433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:21.083611 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 15320
I20260806 01:17:21.083856 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:17:21.084586 26546 external_mini_cluster.cc:1428] Setting key d5950a8000261b0a3b7f7cba3fb425f0
I20260806 01:17:21.091603 15433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35: Bootstrap starting.
I20260806 01:17:21.095278 15433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:21.097091 15433 log.cc:824] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:21.101478 15433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35: No bootstrap required, opened a new log
Aug 06 01:17:21 dist-test-slave-7nm7 krb5kdc[15303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:21.111593 15433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd865854b78493e8c3d52ef1e373a35" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 44931 } }
I20260806 01:17:21.112052 15433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:21.112215 15433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd865854b78493e8c3d52ef1e373a35, State: Initialized, Role: FOLLOWER
I20260806 01:17:21.112968 15433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd865854b78493e8c3d52ef1e373a35" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 44931 } }
I20260806 01:17:21.113291 15433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:17:21.113504 15433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:17:21.113724 15433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:21.114373 15437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:21.086744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40060 (local address 127.25.236.190:44931)
0806 01:17:21.087637 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:21.087649 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:21.087696 (+ 47us) server_negotiation.cc:407] Connection header received
0806 01:17:21.090647 (+ 2951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:21.090692 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:21.091385 (+ 693us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:21.091955 (+ 570us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:21.094056 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:21.096125 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:21.097799 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:21.098731 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:21.098956 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:21.104650 (+ 5694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:21.104662 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:21.104683 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:21.104751 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:21.108678 (+ 3927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:21.109559 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:21.109569 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:21.109578 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:21.109759 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:21.110178 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:21.110182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:21.110184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:21.110971 (+ 787us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:21.111352 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:21.111837 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 01:17:21.112739 (+ 902us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":411,"threads_started":1}
I20260806 01:17:21.117251 15433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd865854b78493e8c3d52ef1e373a35" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 44931 } }
I20260806 01:17:21.117861 15433 leader_election.cc:302] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bd865854b78493e8c3d52ef1e373a35; no voters:
I20260806 01:17:21.118690 15433 leader_election.cc:288] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:17:21.119064 15440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:17:21.120095 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:17:21.121872 15433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:17:21.122611 15440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [term 1 LEADER]: Becoming Leader. State: Replica: 3bd865854b78493e8c3d52ef1e373a35, State: Running, Role: LEADER
I20260806 01:17:21.123337 15440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd865854b78493e8c3d52ef1e373a35" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 44931 } }
I20260806 01:17:21.127348 15442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bd865854b78493e8c3d52ef1e373a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd865854b78493e8c3d52ef1e373a35" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 44931 } } }
I20260806 01:17:21.127894 15442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:21.130554 15443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bd865854b78493e8c3d52ef1e373a35. Latest consensus state: current_term: 1 leader_uuid: "3bd865854b78493e8c3d52ef1e373a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd865854b78493e8c3d52ef1e373a35" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 44931 } } }
I20260806 01:17:21.131062 15443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35 [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:21.131161 15447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:17:21.137300 15447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:17:21.146016 15447 catalog_manager.cc:1347] Generated new cluster ID: 9feefb4f88e94992b9e765703a760bb4
I20260806 01:17:21.146155 15447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:17:21.161662 15447 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:17:21.163327 15447 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:17:21.170343 15447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bd865854b78493e8c3d52ef1e373a35: Generated new TSK 0
I20260806 01:17:21.171170 15447 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:17:21.184129 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 15320
I20260806 01:17:21.185318 15426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:21.278712 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 15320
2026-08-06T01:17:21Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2406 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:17:21 dist-test-slave-7nm7 krb5kdc[15471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:17:21 dist-test-slave-7nm7 krb5kdc[15471](info): set up 2 sockets
Aug 06 01:17:21 dist-test-slave-7nm7 krb5kdc[15471](info): commencing operation
krb5kdc: starting...
W20260806 01:17:22.657079 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.333s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:17:22 dist-test-slave-7nm7 krb5kdc[15471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:17:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:17:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:17:22.973766 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:34371
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:34371
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:23.355537 15490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:23.356015 15490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:23.356228 15490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:23.367363 15490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:17:23.367767 15490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:23.368158 15490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:23.368371 15490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:17:23.368564 15490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:17:23.385263 15490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:34371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:34371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:23.387027 15490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:23.390962 15490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:23.406183 15500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:23.406368 15497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:23.406843 15503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:23.407379 15490 server_base.cc:1034] running on GCE node
I20260806 01:17:23.409503 15490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:23.411561 15490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:23.412832 15490 hybrid_clock.cc:648] HybridClock initialized: now 1785979043412764 us; error 59 us; skew 500 ppm
I20260806 01:17:23.413250 15490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:23 dist-test-slave-7nm7 krb5kdc[15471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979043, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:23.417343 15490 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:17:23.419054 15490 webserver.cc:460] Webserver started at http://127.25.236.190:33505/ using document root <none> and password file <none>
I20260806 01:17:23.419770 15490 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:23.419878 15490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:23.420138 15490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:17:23.423595 15490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "bb58a156f4ed4e1a85ce726195236e9d"
format_stamp: "Formatted at 2026-08-06 01:17:23 on dist-test-slave-7nm7"
server_key: "bbf78cbb4f5627395409ed06623e18da"
server_key_iv: "9073e950fdf61cf77c9c0460a096048f"
server_key_version: "encryptionkey@0"
I20260806 01:17:23.424654 15490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "bb58a156f4ed4e1a85ce726195236e9d"
format_stamp: "Formatted at 2026-08-06 01:17:23 on dist-test-slave-7nm7"
server_key: "bbf78cbb4f5627395409ed06623e18da"
server_key_iv: "9073e950fdf61cf77c9c0460a096048f"
server_key_version: "encryptionkey@0"
I20260806 01:17:23.431025 15490 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 01:17:23.434643 15513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:23.436074 15490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:17:23.436280 15490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "bb58a156f4ed4e1a85ce726195236e9d"
format_stamp: "Formatted at 2026-08-06 01:17:23 on dist-test-slave-7nm7"
server_key: "bbf78cbb4f5627395409ed06623e18da"
server_key_iv: "9073e950fdf61cf77c9c0460a096048f"
server_key_version: "encryptionkey@0"
I20260806 01:17:23.436468 15490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:23.459059 15490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:23.463724 15490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:23.464057 15490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:23.485064 15490 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:34371
I20260806 01:17:23.485110 15592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:34371 every 8 connection(s)
I20260806 01:17:23.486613 15490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:17:23.488046 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 15490
I20260806 01:17:23.488298 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:17:23.488978 26546 external_mini_cluster.cc:1428] Setting key 91dda691657c0d137e23c72c481432f0
I20260806 01:17:23.492751 15594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:23.505122 15594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Bootstrap starting.
Aug 06 01:17:23 dist-test-slave-7nm7 krb5kdc[15471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:23.508911 15594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:23.510038 15594 log.cc:824] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:23.513657 15594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: No bootstrap required, opened a new log
I20260806 01:17:23.515159 15597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:23.490966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.25.236.190:34371)
0806 01:17:23.493701 (+ 2735us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:23.493713 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:23.494292 (+ 579us) server_negotiation.cc:407] Connection header received
0806 01:17:23.496289 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:23.496312 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:23.496706 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:23.497154 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:23.498755 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:23.500126 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:23.501512 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:23.502187 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:23.502451 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:23.506844 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:23.506853 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:23.506866 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:23.506906 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:23.510318 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:23.511257 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:23.511264 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:23.511278 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:23.511511 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:23.511904 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:23.511909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:23.511912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:23.512635 (+ 723us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:23.512990 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:23.513382 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 01:17:23.513811 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2485,"thread_start_us":2172,"threads_started":1}
I20260806 01:17:23.522336 15594 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } }
I20260806 01:17:23.522751 15594 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:23.522882 15594 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb58a156f4ed4e1a85ce726195236e9d, State: Initialized, Role: FOLLOWER
I20260806 01:17:23.523592 15594 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } }
I20260806 01:17:23.523962 15594 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:17:23.524114 15594 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:17:23.524333 15594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:23.526834 15594 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } }
I20260806 01:17:23.527360 15594 leader_election.cc:302] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb58a156f4ed4e1a85ce726195236e9d; no voters:
I20260806 01:17:23.527951 15594 leader_election.cc:288] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:17:23.528131 15599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:17:23.528841 15599 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 1 LEADER]: Becoming Leader. State: Replica: bb58a156f4ed4e1a85ce726195236e9d, State: Running, Role: LEADER
I20260806 01:17:23.529385 15599 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } }
I20260806 01:17:23.530089 15594 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:17:23.532763 15600 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb58a156f4ed4e1a85ce726195236e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } } }
I20260806 01:17:23.533365 15600 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:23.534556 15602 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb58a156f4ed4e1a85ce726195236e9d. Latest consensus state: current_term: 1 leader_uuid: "bb58a156f4ed4e1a85ce726195236e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } } }
I20260806 01:17:23.534917 15610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:17:23.534919 15602 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:23.542083 15610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:17:23.552112 15610 catalog_manager.cc:1347] Generated new cluster ID: e1a0704b68344fae913117409674e8aa
I20260806 01:17:23.552253 15610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:17:23.570098 15610 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:17:23.572484 15610 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:17:23.585537 15610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Generated new TSK 0
I20260806 01:17:23.586513 15610 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:17:23.678013 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34371
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:24.067708 15630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:24.068001 15630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:24.068181 15630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:24.075729 15630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:24.075839 15630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:24.076043 15630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:17:24.086170 15630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:24.087441 15630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:24.088800 15630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:24.102463 15640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:24.102411 15641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:24.102484 15643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:24.103279 15630 server_base.cc:1034] running on GCE node
I20260806 01:17:24.103865 15630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:24.104816 15630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:24.106082 15630 hybrid_clock.cc:648] HybridClock initialized: now 1785979044106023 us; error 70 us; skew 500 ppm
I20260806 01:17:24.106451 15630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:24 dist-test-slave-7nm7 krb5kdc[15471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979044, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:24.113477 15630 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:17:24.114995 15630 webserver.cc:460] Webserver started at http://127.25.236.129:43949/ using document root <none> and password file <none>
I20260806 01:17:24.115633 15630 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:24.115736 15630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:24.115979 15630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:17:24.118846 15630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "b6b160dbc174478a96c0f7c5cc342e98"
format_stamp: "Formatted at 2026-08-06 01:17:24 on dist-test-slave-7nm7"
server_key: "31b0883aa94446e1e2447469283e8607"
server_key_iv: "81cf14ad9375f31a7721ac03df5db2db"
server_key_version: "encryptionkey@0"
I20260806 01:17:24.119551 15630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "b6b160dbc174478a96c0f7c5cc342e98"
format_stamp: "Formatted at 2026-08-06 01:17:24 on dist-test-slave-7nm7"
server_key: "31b0883aa94446e1e2447469283e8607"
server_key_iv: "81cf14ad9375f31a7721ac03df5db2db"
server_key_version: "encryptionkey@0"
I20260806 01:17:24.124601 15630 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 01:17:24.128185 15655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:24.129611 15630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:17:24.129806 15630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "b6b160dbc174478a96c0f7c5cc342e98"
format_stamp: "Formatted at 2026-08-06 01:17:24 on dist-test-slave-7nm7"
server_key: "31b0883aa94446e1e2447469283e8607"
server_key_iv: "81cf14ad9375f31a7721ac03df5db2db"
server_key_version: "encryptionkey@0"
I20260806 01:17:24.130069 15630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:24.151369 15630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:24.156167 15630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:24.156498 15630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:24.157686 15630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:24.159198 15630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:24.159278 15630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:24.159407 15630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:24.159474 15630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:24.197863 15630 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:46883
I20260806 01:17:24.197964 15823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:46883 every 8 connection(s)
I20260806 01:17:24.199234 15630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:17:24.206218 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 15630
I20260806 01:17:24.206523 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:17:24.207177 26546 external_mini_cluster.cc:1428] Setting key 1b9aa210836e6ccbc86e5e430214ac2d
Aug 06 01:17:24 dist-test-slave-7nm7 krb5kdc[15471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979044, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:24.223133 15827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:24.201997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:57023 (local address 127.25.236.190:34371)
0806 01:17:24.202579 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:24.202583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:24.203405 (+ 822us) server_negotiation.cc:407] Connection header received
0806 01:17:24.205382 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:24.205389 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:24.205482 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:24.205605 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:24.207322 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:24.208220 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:24.210069 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:24.210591 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:24.210738 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:24.215401 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:24.215410 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:24.215417 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:24.215461 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:24.218909 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:24.219988 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:24.220000 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:24.220007 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:24.220178 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:24.220701 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:24.220708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:24.220713 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:24.221073 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:24.221236 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:24.222491 (+ 1255us) server_negotiation.cc:299] Negotiation successful
0806 01:17:24.222749 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":367,"threads_started":1}
I20260806 01:17:24.224211 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:24.202466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:57023)
0806 01:17:24.203235 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:24.203273 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:24.205002 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:24.205855 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:24.205863 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:24.206509 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:24.207139 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:24.207156 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:24.208409 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:24.208415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:24.209884 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:24.209907 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:24.210835 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:24.210844 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:24.211086 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:24.211812 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:24.211839 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:24.215135 (+ 3296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:24.219078 (+ 3943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:24.219091 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:24.219113 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:24.219798 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:24.220302 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:24.220310 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:24.220316 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:24.220573 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:24.221231 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:24.221242 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:24.221727 (+ 485us) client_negotiation.cc:769] Sending connection context
0806 01:17:24.222066 (+ 339us) client_negotiation.cc:240] Negotiation successful
0806 01:17:24.223064 (+ 998us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":301,"threads_started":1}
I20260806 01:17:24.226223 15825 heartbeater.cc:347] Connected to a master server at 127.25.236.190:34371
I20260806 01:17:24.226883 15825 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:24.227842 15825 heartbeater.cc:511] Master 127.25.236.190:34371 requested a full tablet report, sending...
I20260806 01:17:24.231343 15538 ts_manager.cc:194] Registered new tserver with Master: b6b160dbc174478a96c0f7c5cc342e98 (127.25.236.129:46883)
I20260806 01:17:24.233055 15538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:57023
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:17:24.281519 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34371
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:24.653965 15832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:24.654467 15832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:24.654667 15832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:24.662662 15832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:24.662848 15832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:24.663049 15832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:17:24.673969 15832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:24.675622 15832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:24.676966 15832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:24.691977 15839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:24.692014 15843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:24.692060 15840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:24.693755 15832 server_base.cc:1034] running on GCE node
I20260806 01:17:24.694880 15832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:24.696081 15832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:24.697311 15832 hybrid_clock.cc:648] HybridClock initialized: now 1785979044697270 us; error 45 us; skew 500 ppm
I20260806 01:17:24.697775 15832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:24 dist-test-slave-7nm7 krb5kdc[15471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979044, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:24.701421 15832 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:17:24.703315 15832 webserver.cc:460] Webserver started at http://127.25.236.130:45817/ using document root <none> and password file <none>
I20260806 01:17:24.704063 15832 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:24.704222 15832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:24.704545 15832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:17:24.707870 15832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "f06a395314b54e85a5ae33591c419e48"
format_stamp: "Formatted at 2026-08-06 01:17:24 on dist-test-slave-7nm7"
server_key: "2647ede744b85f4e3e789eab15313028"
server_key_iv: "6898e73898df6e9d94a822fbcd273e75"
server_key_version: "encryptionkey@0"
I20260806 01:17:24.708695 15832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "f06a395314b54e85a5ae33591c419e48"
format_stamp: "Formatted at 2026-08-06 01:17:24 on dist-test-slave-7nm7"
server_key: "2647ede744b85f4e3e789eab15313028"
server_key_iv: "6898e73898df6e9d94a822fbcd273e75"
server_key_version: "encryptionkey@0"
I20260806 01:17:24.715593 15832 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 01:17:24.719501 15857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:24.721199 15832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:17:24.721410 15832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "f06a395314b54e85a5ae33591c419e48"
format_stamp: "Formatted at 2026-08-06 01:17:24 on dist-test-slave-7nm7"
server_key: "2647ede744b85f4e3e789eab15313028"
server_key_iv: "6898e73898df6e9d94a822fbcd273e75"
server_key_version: "encryptionkey@0"
I20260806 01:17:24.721642 15832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:24.746436 15832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:24.752300 15832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:24.752645 15832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:24.753700 15832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:24.755249 15832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:24.755378 15832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:24.755510 15832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:24.755568 15832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:24.793726 16014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:41607 every 8 connection(s)
I20260806 01:17:24.793771 15832 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:41607
I20260806 01:17:24.795816 15832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:17:24.796947 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 15832
I20260806 01:17:24.797189 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:17:24.797819 26546 external_mini_cluster.cc:1428] Setting key 0c6dc7cd6e9275641452b4813f1b1a02
Aug 06 01:17:24 dist-test-slave-7nm7 krb5kdc[15471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979044, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:24.822777 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:24.799250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:51463 (local address 127.25.236.190:34371)
0806 01:17:24.799725 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:24.799729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:24.800794 (+ 1065us) server_negotiation.cc:407] Connection header received
0806 01:17:24.803896 (+ 3102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:24.803901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:24.803975 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:24.804077 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:24.806467 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:24.807471 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:24.810451 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:24.810904 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:24.811029 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:24.816099 (+ 5070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:24.816104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:24.816108 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:24.816144 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:24.819222 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:24.820213 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:24.820221 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:24.820227 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:24.820386 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:24.820857 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:24.820862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:24.820866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:24.821205 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:24.821381 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:24.822132 (+ 751us) server_negotiation.cc:299] Negotiation successful
0806 01:17:24.822397 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":263,"threads_started":1}
I20260806 01:17:24.823815 16018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:24.799695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:51463)
0806 01:17:24.800616 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:24.800668 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:24.803523 (+ 2855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:24.804370 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:24.804383 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:24.805240 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:24.806136 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:24.806166 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:24.807735 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:24.807743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:24.809975 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:24.809995 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:24.811180 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:24.811191 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:24.811572 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:24.812443 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:24.812485 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:24.815791 (+ 3306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:24.819448 (+ 3657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:24.819459 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:24.819482 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:24.820075 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:24.820523 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:24.820526 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:24.820529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:24.820727 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:24.821360 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:24.821369 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:24.821753 (+ 384us) client_negotiation.cc:769] Sending connection context
0806 01:17:24.822033 (+ 280us) client_negotiation.cc:240] Negotiation successful
0806 01:17:24.822640 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":401,"threads_started":1}
I20260806 01:17:24.825690 16015 heartbeater.cc:347] Connected to a master server at 127.25.236.190:34371
I20260806 01:17:24.826515 16015 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:24.827723 16015 heartbeater.cc:511] Master 127.25.236.190:34371 requested a full tablet report, sending...
I20260806 01:17:24.830127 15538 ts_manager.cc:194] Registered new tserver with Master: f06a395314b54e85a5ae33591c419e48 (127.25.236.130:41607)
I20260806 01:17:24.832306 15538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:51463
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:17:24.878829 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34371
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:17:25.237219 15825 heartbeater.cc:503] Master 127.25.236.190:34371 was elected leader, sending a full tablet report...
W20260806 01:17:25.276598 16021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:25.277043 16021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:25.277248 16021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:25.285864 16021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:25.286001 16021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:25.286444 16021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:17:25.298812 16021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:34371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:25.300379 16021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:25.302083 16021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:25.316833 16028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:25.316883 16026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:25.316906 16030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:25.317987 16021 server_base.cc:1034] running on GCE node
I20260806 01:17:25.318671 16021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:25.319643 16021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:25.320858 16021 hybrid_clock.cc:648] HybridClock initialized: now 1785979045320792 us; error 58 us; skew 500 ppm
I20260806 01:17:25.321242 16021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:25 dist-test-slave-7nm7 krb5kdc[15471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979045, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:25.324790 16021 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:17:25.327263 16021 webserver.cc:460] Webserver started at http://127.25.236.131:35157/ using document root <none> and password file <none>
I20260806 01:17:25.328191 16021 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:25.328656 16021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:25.329209 16021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:17:25.332748 16021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef"
format_stamp: "Formatted at 2026-08-06 01:17:25 on dist-test-slave-7nm7"
server_key: "2f266af31248d6d3ee68979c7c2e58c7"
server_key_iv: "4ed8c76c43e3952509e828d223a2e184"
server_key_version: "encryptionkey@0"
I20260806 01:17:25.333820 16021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef"
format_stamp: "Formatted at 2026-08-06 01:17:25 on dist-test-slave-7nm7"
server_key: "2f266af31248d6d3ee68979c7c2e58c7"
server_key_iv: "4ed8c76c43e3952509e828d223a2e184"
server_key_version: "encryptionkey@0"
I20260806 01:17:25.340199 16021 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 01:17:25.344657 16042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:25.346829 16021 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:17:25.347177 16021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef"
format_stamp: "Formatted at 2026-08-06 01:17:25 on dist-test-slave-7nm7"
server_key: "2f266af31248d6d3ee68979c7c2e58c7"
server_key_iv: "4ed8c76c43e3952509e828d223a2e184"
server_key_version: "encryptionkey@0"
I20260806 01:17:25.347519 16021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:25.371647 16021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:25.377350 16021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:25.377700 16021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:25.378836 16021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:25.380292 16021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:25.380373 16021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:25.380508 16021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:25.380561 16021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:25.416742 16021 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:43555
I20260806 01:17:25.416831 16200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:43555 every 8 connection(s)
I20260806 01:17:25.418057 16021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:17:25.426160 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 16021
I20260806 01:17:25.426447 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:17:25.427047 26546 external_mini_cluster.cc:1428] Setting key 050c40d93862fcf9c442bdb6560472ed
Aug 06 01:17:25 dist-test-slave-7nm7 krb5kdc[15471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979045, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:25.439831 16203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.420683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:44707 (local address 127.25.236.190:34371)
0806 01:17:25.421183 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:25.421187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:25.422053 (+ 866us) server_negotiation.cc:407] Connection header received
0806 01:17:25.424038 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:25.424043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:25.424149 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:25.424277 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:25.425879 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.426887 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.428751 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.429272 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.429411 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.433121 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:25.433128 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:25.433133 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:25.433178 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:25.436161 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:25.436962 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:25.436969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:25.436972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:25.437096 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:25.437544 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:25.437549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:25.437552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:25.437839 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:25.438013 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:25.438866 (+ 853us) server_negotiation.cc:299] Negotiation successful
0806 01:17:25.439089 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":277,"threads_started":1}
I20260806 01:17:25.440896 16204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.421128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:44707)
0806 01:17:25.421885 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:25.421931 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:25.423754 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:25.424518 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:25.424525 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:25.425056 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:25.425648 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:25.425666 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.427039 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.427043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:25.428601 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:25.428613 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.429507 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.429514 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:25.429754 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.430695 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:25.430721 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:25.432912 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:25.436310 (+ 3398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:25.436319 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:25.436335 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:25.436819 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:25.437228 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:25.437233 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:25.437237 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:25.437424 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:25.437977 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:25.437985 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:25.438452 (+ 467us) client_negotiation.cc:769] Sending connection context
0806 01:17:25.438755 (+ 303us) client_negotiation.cc:240] Negotiation successful
0806 01:17:25.439678 (+ 923us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":297,"threads_started":1}
I20260806 01:17:25.442914 16201 heartbeater.cc:347] Connected to a master server at 127.25.236.190:34371
I20260806 01:17:25.443253 16201 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:25.444021 16201 heartbeater.cc:511] Master 127.25.236.190:34371 requested a full tablet report, sending...
I20260806 01:17:25.445542 15534 ts_manager.cc:194] Registered new tserver with Master: 7ff69b6c87cc439c83f5a5a4dbd6f5ef (127.25.236.131:43555)
I20260806 01:17:25.446602 15534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:44707
I20260806 01:17:25.456084 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:17:25.474416 16203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.460910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47070 (local address 127.25.236.190:34371)
0806 01:17:25.461142 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:25.461147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:25.461524 (+ 377us) server_negotiation.cc:407] Connection header received
0806 01:17:25.461705 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:25.461710 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:25.461777 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:25.461876 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:25.463475 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.464370 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.465843 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.466284 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.466383 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.468842 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:25.468846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:25.468850 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:25.468879 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:25.471645 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:25.472444 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:25.472448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:25.472450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:25.472568 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:25.472987 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:25.472991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:25.472993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:25.473354 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:25.473550 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:25.473878 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 01:17:25.474070 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 01:17:25.479046 15534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:17:25.486387 15534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:17:25.513808 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.501211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.25.236.131:43555)
0806 01:17:25.503746 (+ 2535us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:25.503757 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:25.505325 (+ 1568us) server_negotiation.cc:407] Connection header received
0806 01:17:25.505437 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:25.505445 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:25.505697 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:25.506643 (+ 946us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:25.506867 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.507907 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.511085 (+ 3178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.512302 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.512538 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.512728 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:25.513240 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 01:17:25.513420 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2321,"thread_start_us":2140,"threads_started":1}
I20260806 01:17:25.514884 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.500945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:43555 (local address 127.0.0.1:48830)
0806 01:17:25.504875 (+ 3930us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:25.504917 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:25.505111 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:25.505934 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:25.505944 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:25.505975 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:25.506476 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:25.506487 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.508410 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.508415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:25.510896 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:25.510910 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.512529 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.512536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:25.512755 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.512827 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 01:17:25.512982 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 01:17:25.514464 (+ 1482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3659,"thread_start_us":3516,"threads_started":1}
I20260806 01:17:25.519836 16108 tablet_service.cc:1467] Processing CreateTablet for tablet e98d4b218bbe46f8bcc49ce04cfb5e0f (DEFAULT_TABLE table=test-table [id=d6a9a105a59748b69fc980c3316eb32b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:17:25.521389 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.501337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:41607 (local address 127.0.0.1:48428)
0806 01:17:25.512389 (+ 11052us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:25.512414 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:25.513255 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:25.514619 (+ 1364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:25.514623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:25.514653 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:25.515120 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:25.515134 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.516912 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.516920 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:25.519012 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:25.519027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.520344 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.520351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:25.520706 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.520743 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:17:25.520840 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 01:17:25.520984 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10887,"mutex_wait_us":26,"thread_start_us":10753,"threads_started":1}
I20260806 01:17:25.521432 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.501693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.25.236.130:41607)
0806 01:17:25.503145 (+ 1452us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:25.503157 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:25.512568 (+ 9411us) server_negotiation.cc:407] Connection header received
0806 01:17:25.513516 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:25.513529 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:25.513885 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:25.514102 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:25.515362 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.516775 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.519192 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.520115 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.520310 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.520497 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:25.520927 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 01:17:25.521112 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1202,"thread_start_us":1039,"threads_started":1}
I20260806 01:17:25.521498 16108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e98d4b218bbe46f8bcc49ce04cfb5e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:25.523337 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.502365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46883 (local address 127.0.0.1:57856)
0806 01:17:25.515200 (+ 12835us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:25.515221 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:25.515351 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:25.515914 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:25.515918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:25.515943 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:25.516304 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:25.516314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.518478 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.518483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:25.520583 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:25.520599 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.522498 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.522508 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:25.522686 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.522725 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 01:17:25.522813 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 01:17:25.522936 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12645,"mutex_wait_us":4,"thread_start_us":10544,"threads_started":1}
I20260806 01:17:25.524425 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.502615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57856 (local address 127.25.236.129:46883)
0806 01:17:25.514787 (+ 12172us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:25.514801 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:25.515314 (+ 513us) server_negotiation.cc:407] Connection header received
0806 01:17:25.515552 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:25.515560 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:25.515736 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:25.515923 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:25.516519 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.518090 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.520746 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.522288 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.522493 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.523865 (+ 1372us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:25.523939 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 01:17:25.524100 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11986,"thread_start_us":11795,"threads_started":1}
I20260806 01:17:25.525709 15917 tablet_service.cc:1467] Processing CreateTablet for tablet e98d4b218bbe46f8bcc49ce04cfb5e0f (DEFAULT_TABLE table=test-table [id=d6a9a105a59748b69fc980c3316eb32b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:17:25.527194 15917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e98d4b218bbe46f8bcc49ce04cfb5e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:25.528025 15731 tablet_service.cc:1467] Processing CreateTablet for tablet e98d4b218bbe46f8bcc49ce04cfb5e0f (DEFAULT_TABLE table=test-table [id=d6a9a105a59748b69fc980c3316eb32b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:17:25.530431 15731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e98d4b218bbe46f8bcc49ce04cfb5e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:25.534832 16227 tablet_bootstrap.cc:492] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Bootstrap starting.
I20260806 01:17:25.538739 16227 tablet_bootstrap.cc:654] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:25.539794 16227 log.cc:824] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:25.542870 16232 tablet_bootstrap.cc:492] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48: Bootstrap starting.
I20260806 01:17:25.543349 16227 tablet_bootstrap.cc:492] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: No bootstrap required, opened a new log
I20260806 01:17:25.543641 16227 ts_tablet_manager.cc:1397] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260806 01:17:25.545176 16234 tablet_bootstrap.cc:492] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98: Bootstrap starting.
I20260806 01:17:25.548286 16232 tablet_bootstrap.cc:654] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:25.548724 16234 tablet_bootstrap.cc:654] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:25.549657 16232 log.cc:824] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:25.549891 16234 log.cc:824] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:25.552901 16232 tablet_bootstrap.cc:492] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48: No bootstrap required, opened a new log
I20260806 01:17:25.553212 16232 ts_tablet_manager.cc:1397] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260806 01:17:25.553771 16234 tablet_bootstrap.cc:492] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98: No bootstrap required, opened a new log
I20260806 01:17:25.554037 16234 ts_tablet_manager.cc:1397] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260806 01:17:25.554443 16227 raft_consensus.cc:348] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:25.554922 16227 raft_consensus.cc:374] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:25.555027 16227 raft_consensus.cc:729] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ff69b6c87cc439c83f5a5a4dbd6f5ef, State: Initialized, Role: FOLLOWER
I20260806 01:17:25.555675 16227 consensus_queue.cc:260] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:25.558746 16201 heartbeater.cc:503] Master 127.25.236.190:34371 was elected leader, sending a full tablet report...
I20260806 01:17:25.559542 16227 ts_tablet_manager.cc:1428] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260806 01:17:25.564349 16232 raft_consensus.cc:348] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:25.564878 16232 raft_consensus.cc:374] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:25.564987 16232 raft_consensus.cc:729] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06a395314b54e85a5ae33591c419e48, State: Initialized, Role: FOLLOWER
I20260806 01:17:25.564844 16234 raft_consensus.cc:348] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:25.565281 16234 raft_consensus.cc:374] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:25.565388 16234 raft_consensus.cc:729] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b160dbc174478a96c0f7c5cc342e98, State: Initialized, Role: FOLLOWER
I20260806 01:17:25.565668 16232 consensus_queue.cc:260] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:25.565968 16234 consensus_queue.cc:260] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:25.567416 16015 heartbeater.cc:503] Master 127.25.236.190:34371 was elected leader, sending a full tablet report...
I20260806 01:17:25.567826 16234 ts_tablet_manager.cc:1428] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260806 01:17:25.569010 16232 ts_tablet_manager.cc:1428] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48: Time spent starting tablet: real 0.016s user 0.007s sys 0.007s
I20260806 01:17:25.628250 16239 raft_consensus.cc:482] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:17:25.628616 16239 raft_consensus.cc:504] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:25.631462 16239 leader_election.cc:288] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ff69b6c87cc439c83f5a5a4dbd6f5ef (127.25.236.131:43555), b6b160dbc174478a96c0f7c5cc342e98 (127.25.236.129:46883)
I20260806 01:17:25.643028 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.632130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:38683 (local address 127.25.236.131:43555)
0806 01:17:25.632425 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:25.632433 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:25.634055 (+ 1622us) server_negotiation.cc:407] Connection header received
0806 01:17:25.634867 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:25.634875 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:25.634990 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:25.635076 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:25.636543 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.637603 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.640402 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.641719 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.641879 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.641959 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:25.642621 (+ 662us) server_negotiation.cc:299] Negotiation successful
0806 01:17:25.642768 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 01:17:25.643033 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.632964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:47545 (local address 127.25.236.129:46883)
0806 01:17:25.633244 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:25.633253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:25.634450 (+ 1197us) server_negotiation.cc:407] Connection header received
0806 01:17:25.634887 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:25.634895 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:25.635069 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:25.635175 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:25.636208 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.637327 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.640390 (+ 3063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.641701 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.641879 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.641959 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:25.642623 (+ 664us) server_negotiation.cc:299] Negotiation successful
0806 01:17:25.642768 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 01:17:25.643240 16242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.631820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:43555 (local address 127.25.236.130:38683)
0806 01:17:25.633925 (+ 2105us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:25.633955 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:25.634524 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:25.635535 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:25.635548 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:25.635629 (+ 81us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:25.636342 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:25.636355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.637762 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.637767 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:25.640196 (+ 2429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:25.640206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.642069 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.642076 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:25.642389 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.642436 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 01:17:25.642535 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 01:17:25.642714 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1914,"mutex_wait_us":267,"thread_start_us":1917,"threads_started":1}
I20260806 01:17:25.643268 16241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.632282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46883 (local address 127.25.236.130:47545)
0806 01:17:25.634279 (+ 1997us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:25.634314 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:25.634598 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:25.635390 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:25.635398 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:25.635430 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:25.635989 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:25.636005 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.637483 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.637488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:25.640189 (+ 2701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:25.640206 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.642069 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.642076 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:25.642377 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.642429 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 01:17:25.642535 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 01:17:25.642712 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1698,"mutex_wait_us":104,"spinlock_wait_cycles":26368,"thread_start_us":1167,"threads_started":1}
I20260806 01:17:25.644521 16135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e98d4b218bbe46f8bcc49ce04cfb5e0f" candidate_uuid: "f06a395314b54e85a5ae33591c419e48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" is_pre_election: true
I20260806 01:17:25.644608 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e98d4b218bbe46f8bcc49ce04cfb5e0f" candidate_uuid: "f06a395314b54e85a5ae33591c419e48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6b160dbc174478a96c0f7c5cc342e98" is_pre_election: true
I20260806 01:17:25.645241 15758 raft_consensus.cc:2454] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f06a395314b54e85a5ae33591c419e48 in term 0.
I20260806 01:17:25.645094 16135 raft_consensus.cc:2454] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f06a395314b54e85a5ae33591c419e48 in term 0.
I20260806 01:17:25.646476 15861 leader_election.cc:302] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6b160dbc174478a96c0f7c5cc342e98, f06a395314b54e85a5ae33591c419e48; no voters:
I20260806 01:17:25.646981 16239 raft_consensus.cc:2781] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:17:25.647404 16239 raft_consensus.cc:482] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:17:25.647574 16239 raft_consensus.cc:3037] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:25.650219 16239 raft_consensus.cc:504] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:25.651070 16239 leader_election.cc:288] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [CANDIDATE]: Term 1 election: Requested vote from peers 7ff69b6c87cc439c83f5a5a4dbd6f5ef (127.25.236.131:43555), b6b160dbc174478a96c0f7c5cc342e98 (127.25.236.129:46883)
I20260806 01:17:25.651592 16135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e98d4b218bbe46f8bcc49ce04cfb5e0f" candidate_uuid: "f06a395314b54e85a5ae33591c419e48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef"
I20260806 01:17:25.651726 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e98d4b218bbe46f8bcc49ce04cfb5e0f" candidate_uuid: "f06a395314b54e85a5ae33591c419e48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6b160dbc174478a96c0f7c5cc342e98"
I20260806 01:17:25.651813 16135 raft_consensus.cc:3037] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:25.652004 15758 raft_consensus.cc:3037] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:25.654683 15758 raft_consensus.cc:2454] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f06a395314b54e85a5ae33591c419e48 in term 1.
I20260806 01:17:25.655087 16135 raft_consensus.cc:2454] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f06a395314b54e85a5ae33591c419e48 in term 1.
I20260806 01:17:25.655246 15861 leader_election.cc:302] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6b160dbc174478a96c0f7c5cc342e98, f06a395314b54e85a5ae33591c419e48; no voters:
I20260806 01:17:25.655642 16239 raft_consensus.cc:2781] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:17:25.656175 16239 raft_consensus.cc:686] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [term 1 LEADER]: Becoming Leader. State: Replica: f06a395314b54e85a5ae33591c419e48, State: Running, Role: LEADER
I20260806 01:17:25.656724 16239 consensus_queue.cc:237] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:25.662127 15538 catalog_manager.cc:5495] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 reported cstate change: term changed from 0 to 1, leader changed from <none> to f06a395314b54e85a5ae33591c419e48 (127.25.236.130). New cstate: current_term: 1 leader_uuid: "f06a395314b54e85a5ae33591c419e48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } health_report { overall_health: UNKNOWN } } }
W20260806 01:17:25.669982 16202 tablet.cc:2369] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:17:25.702106 15826 tablet.cc:2369] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:17:25.717921 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.710344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48434 (local address 127.25.236.130:41607)
0806 01:17:25.710757 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:25.710762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:25.710783 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:17:25.711045 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:25.711050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:25.711171 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:25.711326 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:17:25.712007 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.712815 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.715212 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.715631 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.715735 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.716231 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:17:25.716756 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:17:25.716868 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:25.717251 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 01:17:25.717442 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233}
I20260806 01:17:25.727260 15758 raft_consensus.cc:1261] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98 [term 1 FOLLOWER]: Refusing update from remote peer f06a395314b54e85a5ae33591c419e48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:17:25.727262 16135 raft_consensus.cc:1261] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 1 FOLLOWER]: Refusing update from remote peer f06a395314b54e85a5ae33591c419e48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:17:25.729276 16244 consensus_queue.cc:1035] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:17:25.729748 16239 consensus_queue.cc:1035] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:17:25.749063 16254 mvcc.cc:205] Tried to move back new op lower bound from 7315370171286974464 to 7315370171014516736. Current Snapshot: MvccSnapshot[applied={T|T < 7315370171286974464}]
I20260806 01:17:25.759847 15538 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:17:25.760161 15538 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:17:25.764166 15538 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Sending DeleteTablet for 3 replicas of tablet e98d4b218bbe46f8bcc49ce04cfb5e0f
I20260806 01:17:25.765408 15917 tablet_service.cc:1514] Processing DeleteTablet for tablet e98d4b218bbe46f8bcc49ce04cfb5e0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:17:25 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:48428
I20260806 01:17:25.765401 15731 tablet_service.cc:1514] Processing DeleteTablet for tablet e98d4b218bbe46f8bcc49ce04cfb5e0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:17:25 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:57856
I20260806 01:17:25.765399 16108 tablet_service.cc:1514] Processing DeleteTablet for tablet e98d4b218bbe46f8bcc49ce04cfb5e0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:17:25 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:48830
I20260806 01:17:25.769945 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 15630
I20260806 01:17:25.777772 16270 tablet_replica.cc:330] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: stopping tablet replica
I20260806 01:17:25.777956 16268 tablet_replica.cc:330] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98: stopping tablet replica
I20260806 01:17:25.778359 16270 raft_consensus.cc:2229] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:17:25.778457 16268 raft_consensus.cc:2229] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:17:25.778930 16270 raft_consensus.cc:2258] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:17:25.778931 16268 raft_consensus.cc:2258] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:17:25.781536 16268 ts_tablet_manager.cc:1905] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:17:25.781538 16270 ts_tablet_manager.cc:1905] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:17:25.782521 16269 tablet_replica.cc:330] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48: stopping tablet replica
I20260806 01:17:25.782961 16269 raft_consensus.cc:2229] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:17:25.783633 16269 raft_consensus.cc:2258] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:17:25.788591 16270 ts_tablet_manager.cc:1918] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:17:25.788748 16270 log.cc:1196] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/e98d4b218bbe46f8bcc49ce04cfb5e0f
I20260806 01:17:25.789284 16270 ts_tablet_manager.cc:1939] T e98d4b218bbe46f8bcc49ce04cfb5e0f P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Deleting consensus metadata
I20260806 01:17:25.791190 15514 catalog_manager.cc:4841] TS 7ff69b6c87cc439c83f5a5a4dbd6f5ef (127.25.236.131:43555): tablet e98d4b218bbe46f8bcc49ce04cfb5e0f (table test-table [id=d6a9a105a59748b69fc980c3316eb32b]) successfully deleted
I20260806 01:17:25.791965 16268 ts_tablet_manager.cc:1918] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:17:25.792224 16269 ts_tablet_manager.cc:1905] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:17:25.792270 16268 log.cc:1196] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/e98d4b218bbe46f8bcc49ce04cfb5e0f
Aug 06 01:17:25 dist-test-slave-7nm7 krb5kdc[15471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:17:25.800408 16269 ts_tablet_manager.cc:1918] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:17:25.800593 16269 log.cc:1196] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/e98d4b218bbe46f8bcc49ce04cfb5e0f
I20260806 01:17:25.801232 16269 ts_tablet_manager.cc:1939] T e98d4b218bbe46f8bcc49ce04cfb5e0f P f06a395314b54e85a5ae33591c419e48: Deleting consensus metadata
I20260806 01:17:25.803197 15515 catalog_manager.cc:4841] TS f06a395314b54e85a5ae33591c419e48 (127.25.236.130:41607): tablet e98d4b218bbe46f8bcc49ce04cfb5e0f (table test-table [id=d6a9a105a59748b69fc980c3316eb32b]) successfully deleted
I20260806 01:17:25.803822 16268 ts_tablet_manager.cc:1939] T e98d4b218bbe46f8bcc49ce04cfb5e0f P b6b160dbc174478a96c0f7c5cc342e98: Deleting consensus metadata
I20260806 01:17:25.805626 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.771569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57868 (local address 127.25.236.129:46883)
0806 01:17:25.771768 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:25.771775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:25.785533 (+ 13758us) server_negotiation.cc:407] Connection header received
0806 01:17:25.786058 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:25.786066 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:25.786153 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:25.786256 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:25.788258 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.789418 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.791761 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.792409 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.792545 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.798133 (+ 5588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:25.798148 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:25.798165 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:25.798217 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:25.802123 (+ 3906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:25.803100 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:25.803110 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:25.803113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:25.803300 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:25.803820 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:25.803826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:25.803829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:25.804355 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:25.804700 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:25.805006 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 01:17:25.805241 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:17:25.805716 15516 catalog_manager.cc:4841] TS b6b160dbc174478a96c0f7c5cc342e98 (127.25.236.129:46883): tablet e98d4b218bbe46f8bcc49ce04cfb5e0f (table test-table [id=d6a9a105a59748b69fc980c3316eb32b]) successfully deleted
I20260806 01:17:25.807775 15817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:25.912952 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 15630
I20260806 01:17:25.947760 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 15832
Aug 06 01:17:25 dist-test-slave-7nm7 krb5kdc[15471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:17:25.974036 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:25.948895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48446 (local address 127.25.236.130:41607)
0806 01:17:25.949112 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:25.949124 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:25.949147 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:17:25.949362 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:25.949368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:25.949472 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:25.949596 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:25.950990 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.952152 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.953527 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:25.954193 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:25.954398 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:25.964538 (+ 10140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:25.964562 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:25.964598 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:25.964652 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:25.969426 (+ 4774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:25.970790 (+ 1364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:25.970801 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:25.970805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:25.970986 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:25.971531 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:25.971537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:25.971540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:25.972564 (+ 1024us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:25.973143 (+ 579us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:25.973376 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 01:17:25.973639 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:17:25.975330 16009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:26.106874 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 15832
I20260806 01:17:26.136276 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 16021
Aug 06 01:17:26 dist-test-slave-7nm7 krb5kdc[15471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:17:26.153251 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:26.137072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48836 (local address 127.25.236.131:43555)
0806 01:17:26.137353 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:26.137361 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:26.137417 (+ 56us) server_negotiation.cc:407] Connection header received
0806 01:17:26.137616 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:26.137621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:26.137706 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:26.137803 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:26.139143 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:26.139931 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:26.141276 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:26.141778 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:26.141884 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:26.146977 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:26.146992 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:26.147009 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:26.147046 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:26.150093 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:26.150873 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:26.150880 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:26.150882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:26.151041 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:26.151537 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:26.151541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:26.151543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:26.151924 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:26.152165 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:26.152600 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 01:17:26.152780 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 01:17:26.154023 16195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:26.250726 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 16021
I20260806 01:17:26.277403 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 15490
I20260806 01:17:26.278803 15587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:26.394948 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 15490
I20260806 01:17:26.431196 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:34371
--webserver_interface=127.25.236.190
--webserver_port=33505
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:34371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:26.800283 16278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:26.800653 16278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:26.800830 16278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:26.808400 16278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:17:26.808564 16278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:26.808682 16278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:26.808794 16278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:17:26.808887 16278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:17:26.819617 16278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:34371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:34371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=33505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:26.821008 16278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:26.822391 16278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:26.851361 16290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:26.851544 16293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:26.851364 16291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:26.852330 16278 server_base.cc:1034] running on GCE node
I20260806 01:17:26.853928 16278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:26.856047 16278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:26.857332 16278 hybrid_clock.cc:648] HybridClock initialized: now 1785979046857256 us; error 34 us; skew 500 ppm
I20260806 01:17:26.857777 16278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:26 dist-test-slave-7nm7 krb5kdc[15471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:26.861697 16278 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:17:26.863692 16278 webserver.cc:460] Webserver started at http://127.25.236.190:33505/ using document root <none> and password file <none>
I20260806 01:17:26.864508 16278 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:26.864666 16278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:26.871233 16278 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 01:17:26.874981 16300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:26.876497 16278 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:17:26.876709 16278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "bb58a156f4ed4e1a85ce726195236e9d"
format_stamp: "Formatted at 2026-08-06 01:17:23 on dist-test-slave-7nm7"
server_key: "bbf78cbb4f5627395409ed06623e18da"
server_key_iv: "9073e950fdf61cf77c9c0460a096048f"
server_key_version: "encryptionkey@0"
I20260806 01:17:26.877856 16278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:26.906299 16278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:26.910971 16278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:26.911238 16278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:26.932806 16278 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:34371
I20260806 01:17:26.932824 16369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:34371 every 8 connection(s)
I20260806 01:17:26.934617 16278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:17:26.937269 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 16278
I20260806 01:17:26.938150 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:46883
--local_ip_for_outbound_sockets=127.25.236.129
--tserver_master_addrs=127.25.236.190:34371
--webserver_port=43949
--webserver_interface=127.25.236.129
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:17:26.942304 16373 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:17:26.945706 16373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Bootstrap starting.
I20260806 01:17:26.967546 16373 log.cc:824] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:26.981878 16373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:17:26.982645 16373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Bootstrap complete.
I20260806 01:17:26.992262 16373 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } }
I20260806 01:17:26.993335 16373 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb58a156f4ed4e1a85ce726195236e9d, State: Initialized, Role: FOLLOWER
I20260806 01:17:26.993950 16373 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } }
I20260806 01:17:26.994225 16373 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:17:26.994463 16373 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:17:26.994670 16373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:17:26.999490 16373 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } }
I20260806 01:17:27.000110 16373 leader_election.cc:302] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb58a156f4ed4e1a85ce726195236e9d; no voters:
I20260806 01:17:27.000793 16373 leader_election.cc:288] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 01:17:27.000954 16378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:17:27.003572 16373 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:17:27.008179 16378 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 2 LEADER]: Becoming Leader. State: Replica: bb58a156f4ed4e1a85ce726195236e9d, State: Running, Role: LEADER
I20260806 01:17:27.008982 16378 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } }
I20260806 01:17:27.011900 16379 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bb58a156f4ed4e1a85ce726195236e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } } }
I20260806 01:17:27.012373 16379 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:27.013013 16380 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb58a156f4ed4e1a85ce726195236e9d. Latest consensus state: current_term: 2 leader_uuid: "bb58a156f4ed4e1a85ce726195236e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } } }
I20260806 01:17:27.013350 16380 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:27.017436 16392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:17:27.025770 16392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:17:27.027631 16392 catalog_manager.cc:1259] Loaded cluster ID: e1a0704b68344fae913117409674e8aa
I20260806 01:17:27.027770 16392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:17:27.032292 16392 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 01:17:27.050449 16392 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 01:17:27.059895 16392 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:17:27.061331 16392 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Loaded TSK: 0
I20260806 01:17:27.062592 16392 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 01:17:27.349196 16376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:27.349488 16376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:27.349645 16376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:27.359308 16376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:27.359436 16376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:27.359606 16376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:17:27.370950 16376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:46883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=43949
--tserver_master_addrs=127.25.236.190:34371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:27.372900 16376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:27.375023 16376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:27.392930 16413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:27.393867 16414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:27.394414 16417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:27.394999 16376 server_base.cc:1034] running on GCE node
I20260806 01:17:27.395973 16376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:27.397503 16376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:27.398795 16376 hybrid_clock.cc:648] HybridClock initialized: now 1785979047398711 us; error 82 us; skew 500 ppm
I20260806 01:17:27.399518 16376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:27 dist-test-slave-7nm7 krb5kdc[15471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979047, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:27.404199 16376 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:17:27.406173 16376 webserver.cc:460] Webserver started at http://127.25.236.129:43949/ using document root <none> and password file <none>
I20260806 01:17:27.407174 16376 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:27.407392 16376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:27.415220 16376 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.002s
I20260806 01:17:27.419028 16426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:27.421011 16376 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:17:27.421195 16376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "b6b160dbc174478a96c0f7c5cc342e98"
format_stamp: "Formatted at 2026-08-06 01:17:24 on dist-test-slave-7nm7"
server_key: "31b0883aa94446e1e2447469283e8607"
server_key_iv: "81cf14ad9375f31a7721ac03df5db2db"
server_key_version: "encryptionkey@0"
I20260806 01:17:27.422617 16376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:27.467527 16376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:27.474023 16376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:27.474378 16376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:27.475564 16376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:27.477290 16376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:27.477401 16376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:27.477489 16376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:27.477568 16376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:27.516343 16376 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:46883
I20260806 01:17:27.516436 16587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:46883 every 8 connection(s)
I20260806 01:17:27.518038 16376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:17:27.522718 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 16376
I20260806 01:17:27.523362 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:41607
--local_ip_for_outbound_sockets=127.25.236.130
--tserver_master_addrs=127.25.236.190:34371
--webserver_port=45817
--webserver_interface=127.25.236.130
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:17:27 dist-test-slave-7nm7 krb5kdc[15471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979047, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:27.543597 16590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:27.520918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:54799 (local address 127.25.236.190:34371)
0806 01:17:27.521512 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:27.521530 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:27.522015 (+ 485us) server_negotiation.cc:407] Connection header received
0806 01:17:27.524231 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:27.524242 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:27.524539 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:27.524887 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:27.526424 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:27.527775 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:27.529477 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:27.530226 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:27.530560 (+ 334us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:27.535017 (+ 4457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:27.535025 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:27.535039 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:27.535077 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:27.538180 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:27.539525 (+ 1345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:27.539533 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:27.539539 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:27.539729 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:27.540438 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:27.540442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:27.540444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:27.540955 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:27.541286 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:27.542089 (+ 803us) server_negotiation.cc:299] Negotiation successful
0806 01:17:27.542477 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":256,"threads_started":1}
I20260806 01:17:27.544049 16591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:27.521095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:54799)
0806 01:17:27.521845 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:27.521888 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:27.523737 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:27.525051 (+ 1314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:27.525061 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:27.525659 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:27.526180 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:27.526197 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:27.527952 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:27.527958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:27.529331 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:27.529352 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:27.530597 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:27.530606 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:27.530883 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:27.531719 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:27.531743 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:27.534724 (+ 2981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:27.538383 (+ 3659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:27.538396 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:27.538415 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:27.539068 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:27.539874 (+ 806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:27.539882 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:27.539889 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:27.540293 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:27.541115 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:27.541127 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:27.541582 (+ 455us) client_negotiation.cc:769] Sending connection context
0806 01:17:27.541895 (+ 313us) client_negotiation.cc:240] Negotiation successful
0806 01:17:27.542722 (+ 827us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":326,"threads_started":1}
I20260806 01:17:27.548128 16588 heartbeater.cc:347] Connected to a master server at 127.25.236.190:34371
I20260806 01:17:27.548499 16588 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:27.549288 16588 heartbeater.cc:511] Master 127.25.236.190:34371 requested a full tablet report, sending...
I20260806 01:17:27.552223 16324 ts_manager.cc:194] Registered new tserver with Master: b6b160dbc174478a96c0f7c5cc342e98 (127.25.236.129:46883)
I20260806 01:17:27.554646 16324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:54799
W20260806 01:17:27.881793 16593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:27.882144 16593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:27.882304 16593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:27.890362 16593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:27.890514 16593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:27.890705 16593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:17:27.900717 16593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:41607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=45817
--tserver_master_addrs=127.25.236.190:34371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:27.902007 16593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:27.903402 16593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:27.922767 16602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:27.922689 16603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:27.922747 16605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:27.925730 16593 server_base.cc:1034] running on GCE node
I20260806 01:17:27.926452 16593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:27.927413 16593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:27.928653 16593 hybrid_clock.cc:648] HybridClock initialized: now 1785979047928605 us; error 67 us; skew 500 ppm
I20260806 01:17:27.929008 16593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:27 dist-test-slave-7nm7 krb5kdc[15471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979047, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:27.932519 16593 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:17:27.934093 16593 webserver.cc:460] Webserver started at http://127.25.236.130:45817/ using document root <none> and password file <none>
I20260806 01:17:27.934820 16593 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:27.934933 16593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:27.940366 16593 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 01:17:27.943302 16618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:27.944747 16593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:17:27.944947 16593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "f06a395314b54e85a5ae33591c419e48"
format_stamp: "Formatted at 2026-08-06 01:17:24 on dist-test-slave-7nm7"
server_key: "2647ede744b85f4e3e789eab15313028"
server_key_iv: "6898e73898df6e9d94a822fbcd273e75"
server_key_version: "encryptionkey@0"
I20260806 01:17:27.945955 16593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:27.969214 16593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:27.973376 16593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:27.973654 16593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:27.974725 16593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:27.976082 16593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:27.976222 16593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:27.976364 16593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:27.976455 16593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:28.011914 16593 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:41607
I20260806 01:17:28.012006 16780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:41607 every 8 connection(s)
I20260806 01:17:28.013371 16593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:17:28.019879 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 16593
I20260806 01:17:28.020498 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:43555
--local_ip_for_outbound_sockets=127.25.236.131
--tserver_master_addrs=127.25.236.190:34371
--webserver_port=35157
--webserver_interface=127.25.236.131
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:17:28 dist-test-slave-7nm7 krb5kdc[15471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979047, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:28.035840 16590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.015995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:50123 (local address 127.25.236.190:34371)
0806 01:17:28.016198 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:28.016203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:28.017308 (+ 1105us) server_negotiation.cc:407] Connection header received
0806 01:17:28.019267 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:28.019272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:28.019446 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:28.019577 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:28.021219 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.022448 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.024124 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.024899 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.025049 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.028543 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:28.028553 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:28.028561 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:28.028633 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:28.032208 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:28.033222 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:28.033231 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:28.033234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:28.033396 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:28.033922 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:28.033926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:28.033928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:28.034324 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:28.034443 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:28.035399 (+ 956us) server_negotiation.cc:299] Negotiation successful
0806 01:17:28.035588 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:17:28.037468 16783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.016450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:50123)
0806 01:17:28.017128 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:28.017174 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:28.019020 (+ 1846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:28.019840 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:28.019848 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:28.020387 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:28.021033 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.021051 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.022635 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.022639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.023930 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.023942 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.025143 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.025149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.025352 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.026038 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:28.026064 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:28.028333 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:28.032356 (+ 4023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:28.032369 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:28.032388 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:28.033067 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:28.033532 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:28.033540 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:28.033546 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:28.033798 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:28.034450 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:28.034461 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:28.034968 (+ 507us) client_negotiation.cc:769] Sending connection context
0806 01:17:28.035298 (+ 330us) client_negotiation.cc:240] Negotiation successful
0806 01:17:28.035912 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":266,"threads_started":1}
I20260806 01:17:28.039218 16781 heartbeater.cc:347] Connected to a master server at 127.25.236.190:34371
I20260806 01:17:28.039566 16781 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:28.040340 16781 heartbeater.cc:511] Master 127.25.236.190:34371 requested a full tablet report, sending...
I20260806 01:17:28.042289 16324 ts_manager.cc:194] Registered new tserver with Master: f06a395314b54e85a5ae33591c419e48 (127.25.236.130:41607)
I20260806 01:17:28.043642 16324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:50123
W20260806 01:17:28.372180 16784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:28.372449 16784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:28.372543 16784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:28.379757 16784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:28.379873 16784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:28.379988 16784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:17:28.389930 16784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:43555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=35157
--tserver_master_addrs=127.25.236.190:34371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:28.391245 16784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:28.392472 16784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:28.406065 16792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:28.406905 16798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:28.406991 16784 server_base.cc:1034] running on GCE node
W20260806 01:17:28.407266 16793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:28.408010 16784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:28.409155 16784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:28.410424 16784 hybrid_clock.cc:648] HybridClock initialized: now 1785979048410406 us; error 130 us; skew 500 ppm
I20260806 01:17:28.410776 16784 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:28 dist-test-slave-7nm7 krb5kdc[15471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979048, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:28.414367 16784 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:17:28.415874 16784 webserver.cc:460] Webserver started at http://127.25.236.131:35157/ using document root <none> and password file <none>
I20260806 01:17:28.416507 16784 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:28.416661 16784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:28.422278 16784 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:17:28.425041 16806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:28.426442 16784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:17:28.426630 16784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef"
format_stamp: "Formatted at 2026-08-06 01:17:25 on dist-test-slave-7nm7"
server_key: "2f266af31248d6d3ee68979c7c2e58c7"
server_key_iv: "4ed8c76c43e3952509e828d223a2e184"
server_key_version: "encryptionkey@0"
I20260806 01:17:28.427580 16784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:28.452169 16784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:28.456681 16784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:28.456983 16784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:28.458056 16784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:28.459594 16784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:28.459719 16784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:28.459853 16784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:28.459947 16784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:28.497195 16784 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:43555
I20260806 01:17:28.497241 16961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:43555 every 8 connection(s)
I20260806 01:17:28.498728 16784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:17:28.505942 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 16784
Aug 06 01:17:28 dist-test-slave-7nm7 krb5kdc[15471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979048, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:28.522406 16590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.501420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:53757 (local address 127.25.236.190:34371)
0806 01:17:28.501630 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:28.501634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:28.502800 (+ 1166us) server_negotiation.cc:407] Connection header received
0806 01:17:28.504825 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:28.504833 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:28.504919 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:28.505061 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:28.507068 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.508601 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.510283 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.511117 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.511389 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.515454 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:28.515460 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:28.515463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:28.515495 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:28.518668 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:28.519609 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:28.519615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:28.519618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:28.519753 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:28.520327 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:28.520335 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:28.520339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:28.520709 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:28.520931 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:28.521772 (+ 841us) server_negotiation.cc:299] Negotiation successful
0806 01:17:28.521993 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":83}
I20260806 01:17:28.523307 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.501826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:53757)
0806 01:17:28.502633 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:28.502675 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:28.504508 (+ 1833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:28.505294 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:28.505302 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:28.506074 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:28.506856 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.506874 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.508779 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.508785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.510109 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.510121 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.511664 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.511680 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.511930 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.512701 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:28.512733 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:28.515083 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:28.518834 (+ 3751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:28.518844 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:28.518860 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:28.519487 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:28.519875 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:28.519881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:28.519885 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:28.520166 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:28.520878 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:28.520888 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:28.521329 (+ 441us) client_negotiation.cc:769] Sending connection context
0806 01:17:28.521653 (+ 324us) client_negotiation.cc:240] Negotiation successful
0806 01:17:28.522055 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":326,"threads_started":1}
I20260806 01:17:28.524857 16962 heartbeater.cc:347] Connected to a master server at 127.25.236.190:34371
I20260806 01:17:28.525343 16962 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:28.525383 16965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.507045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.25.236.190:34371)
0806 01:17:28.507743 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:28.507747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:28.508281 (+ 534us) server_negotiation.cc:407] Connection header received
0806 01:17:28.508484 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:28.508489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:28.508560 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:28.508689 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:28.510348 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.511660 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.513381 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.513859 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.514004 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.517707 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:28.517714 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:28.517719 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:28.517755 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:28.521524 (+ 3769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:28.523188 (+ 1664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:28.523193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:28.523196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:28.523388 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:28.523911 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:28.523915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:28.523917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:28.524185 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:28.524438 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:28.524790 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 01:17:28.525010 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":472,"threads_started":1}
I20260806 01:17:28.526167 16962 heartbeater.cc:511] Master 127.25.236.190:34371 requested a full tablet report, sending...
I20260806 01:17:28.527940 16323 ts_manager.cc:194] Registered new tserver with Master: 7ff69b6c87cc439c83f5a5a4dbd6f5ef (127.25.236.131:43555)
I20260806 01:17:28.528986 16323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:53757
I20260806 01:17:28.541272 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:17:28.548702 16965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.542200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47096 (local address 127.25.236.190:34371)
0806 01:17:28.542422 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:28.542426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:28.542970 (+ 544us) server_negotiation.cc:407] Connection header received
0806 01:17:28.543164 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:28.543168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:28.543329 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:28.543471 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:17:28.543879 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.544554 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.546168 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.546855 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.546961 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.547537 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:17:28.548133 (+ 596us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:17:28.548245 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:28.548401 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 01:17:28.548472 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 01:17:28.549805 16323 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:17:28.557224 16323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:17:28.559345 16588 heartbeater.cc:503] Master 127.25.236.190:34371 was elected leader, sending a full tablet report...
I20260806 01:17:28.592506 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.572022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:41607 (local address 127.0.0.1:48462)
0806 01:17:28.580410 (+ 8388us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:28.580466 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:28.580738 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:28.583228 (+ 2490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:28.583235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:28.583265 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:28.583780 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.583799 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.586500 (+ 2701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.586508 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.588681 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.588695 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.591193 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.591215 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.591437 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.591506 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 01:17:28.591664 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 01:17:28.591841 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8019,"mutex_wait_us":260,"thread_start_us":9858,"threads_started":1}
I20260806 01:17:28.592563 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.573044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48848 (local address 127.25.236.131:43555)
0806 01:17:28.583578 (+ 10534us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:28.583586 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:28.583627 (+ 41us) server_negotiation.cc:407] Connection header received
0806 01:17:28.583712 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:28.583716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:28.583878 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:28.584096 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:28.584926 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.586187 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.589771 (+ 3584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.591305 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.591517 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.591686 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:28.592020 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 01:17:28.592197 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10340,"spinlock_wait_cycles":24320,"thread_start_us":10302,"threads_started":1}
I20260806 01:17:28.593245 16979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.572251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:43555 (local address 127.0.0.1:48848)
0806 01:17:28.582310 (+ 10059us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:28.582330 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:28.582469 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:28.584319 (+ 1850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:28.584329 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:28.584357 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:28.584719 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.584727 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.586416 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.586425 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.589231 (+ 2806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.589249 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.591598 (+ 2349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.591605 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.591814 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.591854 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 01:17:28.591962 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 01:17:28.592280 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9814,"mutex_wait_us":4,"thread_start_us":9820,"threads_started":1}
I20260806 01:17:28.593670 16982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.572781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46883 (local address 127.0.0.1:57872)
0806 01:17:28.582399 (+ 9618us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:28.582415 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:28.582516 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:28.584030 (+ 1514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:28.584034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:28.584057 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:28.584366 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.584378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.587030 (+ 2652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.587036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.589637 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.589673 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.592590 (+ 2917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.592598 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.592793 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.592825 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:17:28.592944 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 01:17:28.593124 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9477,"mutex_wait_us":2,"thread_start_us":7108,"threads_started":1}
I20260806 01:17:28.593981 16980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.572248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.25.236.130:41607)
0806 01:17:28.582635 (+ 10387us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:28.582642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:28.582667 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:17:28.582748 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:28.582754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:28.583005 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:28.583225 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:28.584001 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.586303 (+ 2302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.588869 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.590883 (+ 2014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.591078 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.591392 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:28.592633 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0806 01:17:28.593663 (+ 1030us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9447,"thread_start_us":9303,"threads_started":1}
I20260806 01:17:28.594087 16985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.573568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57872 (local address 127.25.236.129:46883)
0806 01:17:28.581511 (+ 7943us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:28.581524 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:28.583509 (+ 1985us) server_negotiation.cc:407] Connection header received
0806 01:17:28.583618 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:28.583627 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:28.583855 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:28.584032 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:28.584578 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.586877 (+ 2299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.589855 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.591635 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.591860 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.592258 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:28.593361 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0806 01:17:28.593558 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7721,"thread_start_us":7551,"threads_started":1}
I20260806 01:17:28.597553 16873 tablet_service.cc:1467] Processing CreateTablet for tablet 5912af177c2d42728f702f34d4df6363 (DEFAULT_TABLE table=test-table [id=e4a7c79700824ed980ba5cb664bd8b89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:17:28.597747 16687 tablet_service.cc:1467] Processing CreateTablet for tablet 5912af177c2d42728f702f34d4df6363 (DEFAULT_TABLE table=test-table [id=e4a7c79700824ed980ba5cb664bd8b89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:17:28.597747 16492 tablet_service.cc:1467] Processing CreateTablet for tablet 5912af177c2d42728f702f34d4df6363 (DEFAULT_TABLE table=test-table [id=e4a7c79700824ed980ba5cb664bd8b89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:17:28.598999 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5912af177c2d42728f702f34d4df6363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:28.599105 16687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5912af177c2d42728f702f34d4df6363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:28.599123 16492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5912af177c2d42728f702f34d4df6363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:28.614856 16986 tablet_bootstrap.cc:492] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Bootstrap starting.
I20260806 01:17:28.614948 16988 tablet_bootstrap.cc:492] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48: Bootstrap starting.
I20260806 01:17:28.616604 16989 tablet_bootstrap.cc:492] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98: Bootstrap starting.
I20260806 01:17:28.618764 16988 tablet_bootstrap.cc:654] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:28.618762 16986 tablet_bootstrap.cc:654] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:28.619496 16989 tablet_bootstrap.cc:654] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:28.620142 16988 log.cc:824] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:28.620142 16986 log.cc:824] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:28.620573 16989 log.cc:824] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:28.623795 16988 tablet_bootstrap.cc:492] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48: No bootstrap required, opened a new log
I20260806 01:17:28.623793 16986 tablet_bootstrap.cc:492] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: No bootstrap required, opened a new log
I20260806 01:17:28.624092 16988 ts_tablet_manager.cc:1397] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.001s
I20260806 01:17:28.624087 16986 ts_tablet_manager.cc:1397] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260806 01:17:28.628665 16989 tablet_bootstrap.cc:492] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98: No bootstrap required, opened a new log
I20260806 01:17:28.628984 16989 ts_tablet_manager.cc:1397] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.008s
I20260806 01:17:28.636974 16988 raft_consensus.cc:348] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:28.636974 16986 raft_consensus.cc:348] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:28.637519 16986 raft_consensus.cc:374] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:28.637519 16988 raft_consensus.cc:374] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:28.637643 16988 raft_consensus.cc:729] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06a395314b54e85a5ae33591c419e48, State: Initialized, Role: FOLLOWER
I20260806 01:17:28.637631 16986 raft_consensus.cc:729] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ff69b6c87cc439c83f5a5a4dbd6f5ef, State: Initialized, Role: FOLLOWER
I20260806 01:17:28.638386 16986 consensus_queue.cc:260] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:28.638386 16988 consensus_queue.cc:260] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:28.639163 16989 raft_consensus.cc:348] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:28.639650 16989 raft_consensus.cc:374] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:28.639767 16989 raft_consensus.cc:729] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b160dbc174478a96c0f7c5cc342e98, State: Initialized, Role: FOLLOWER
I20260806 01:17:28.640442 16781 heartbeater.cc:503] Master 127.25.236.190:34371 was elected leader, sending a full tablet report...
I20260806 01:17:28.640516 16986 ts_tablet_manager.cc:1428] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260806 01:17:28.640412 16989 consensus_queue.cc:260] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:28.640813 16988 ts_tablet_manager.cc:1428] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48: Time spent starting tablet: real 0.017s user 0.010s sys 0.008s
I20260806 01:17:28.641000 16962 heartbeater.cc:503] Master 127.25.236.190:34371 was elected leader, sending a full tablet report...
I20260806 01:17:28.643509 16989 ts_tablet_manager.cc:1428] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260806 01:17:28.694707 17000 raft_consensus.cc:482] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:17:28.695073 17000 raft_consensus.cc:504] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:28.696815 17000 leader_election.cc:288] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ff69b6c87cc439c83f5a5a4dbd6f5ef (127.25.236.131:43555), b6b160dbc174478a96c0f7c5cc342e98 (127.25.236.129:46883)
I20260806 01:17:28.707890 16985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.697392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:33879 (local address 127.25.236.129:46883)
0806 01:17:28.697593 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:28.697600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:28.698921 (+ 1321us) server_negotiation.cc:407] Connection header received
0806 01:17:28.699058 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:28.699063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:28.699142 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:28.699239 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:28.700214 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.701313 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.704450 (+ 3137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.706292 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.706628 (+ 336us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.706694 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:28.707451 (+ 757us) server_negotiation.cc:299] Negotiation successful
0806 01:17:28.707604 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:17:28.707890 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.697372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:54253 (local address 127.25.236.131:43555)
0806 01:17:28.697593 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:28.697600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:28.698707 (+ 1107us) server_negotiation.cc:407] Connection header received
0806 01:17:28.698959 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:28.698965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:28.699057 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:28.699194 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:28.700161 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.701344 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.704450 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.706286 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.706614 (+ 328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.706694 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:28.707451 (+ 757us) server_negotiation.cc:299] Negotiation successful
0806 01:17:28.707602 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 01:17:28.708521 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.697470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:46883 (local address 127.25.236.130:33879)
0806 01:17:28.698802 (+ 1332us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:28.698821 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:28.698940 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:28.699282 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:28.699288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:28.699347 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:28.700035 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.700050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.701485 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.701491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.704240 (+ 2749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.704257 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.706708 (+ 2451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.706716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.706993 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.707207 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 01:17:28.707375 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 01:17:28.707624 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1202,"mutex_wait_us":379,"spinlock_wait_cycles":51584,"thread_start_us":448,"threads_started":1}
I20260806 01:17:28.708537 17005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.697459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:43555 (local address 127.25.236.130:54253)
0806 01:17:28.698521 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:28.698562 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:28.698756 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:28.699282 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:28.699288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:28.699347 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:28.699923 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.699945 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.701485 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.701491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.704241 (+ 2750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:28.704257 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.706708 (+ 2451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.706716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:28.706990 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.707228 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 01:17:28.707375 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 01:17:28.707624 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":372,"spinlock_wait_cycles":17408,"thread_start_us":975,"threads_started":1}
I20260806 01:17:28.710578 16515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5912af177c2d42728f702f34d4df6363" candidate_uuid: "f06a395314b54e85a5ae33591c419e48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6b160dbc174478a96c0f7c5cc342e98" is_pre_election: true
I20260806 01:17:28.710672 16902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5912af177c2d42728f702f34d4df6363" candidate_uuid: "f06a395314b54e85a5ae33591c419e48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" is_pre_election: true
I20260806 01:17:28.711261 16515 raft_consensus.cc:2454] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f06a395314b54e85a5ae33591c419e48 in term 0.
I20260806 01:17:28.711474 16902 raft_consensus.cc:2454] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f06a395314b54e85a5ae33591c419e48 in term 0.
I20260806 01:17:28.712088 16621 leader_election.cc:302] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6b160dbc174478a96c0f7c5cc342e98, f06a395314b54e85a5ae33591c419e48; no voters:
I20260806 01:17:28.712560 17000 raft_consensus.cc:2781] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:17:28.712733 17000 raft_consensus.cc:482] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:17:28.712862 17000 raft_consensus.cc:3037] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:28.715641 17000 raft_consensus.cc:504] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:28.716367 17000 leader_election.cc:288] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [CANDIDATE]: Term 1 election: Requested vote from peers 7ff69b6c87cc439c83f5a5a4dbd6f5ef (127.25.236.131:43555), b6b160dbc174478a96c0f7c5cc342e98 (127.25.236.129:46883)
I20260806 01:17:28.716882 16902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5912af177c2d42728f702f34d4df6363" candidate_uuid: "f06a395314b54e85a5ae33591c419e48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef"
I20260806 01:17:28.716949 16515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5912af177c2d42728f702f34d4df6363" candidate_uuid: "f06a395314b54e85a5ae33591c419e48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6b160dbc174478a96c0f7c5cc342e98"
I20260806 01:17:28.717114 16902 raft_consensus.cc:3037] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:28.717185 16515 raft_consensus.cc:3037] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:28.719548 16902 raft_consensus.cc:2454] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f06a395314b54e85a5ae33591c419e48 in term 1.
I20260806 01:17:28.719547 16515 raft_consensus.cc:2454] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f06a395314b54e85a5ae33591c419e48 in term 1.
I20260806 01:17:28.720064 16619 leader_election.cc:302] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ff69b6c87cc439c83f5a5a4dbd6f5ef, f06a395314b54e85a5ae33591c419e48; no voters:
I20260806 01:17:28.720655 17000 raft_consensus.cc:2781] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:17:28.721120 17000 raft_consensus.cc:686] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [term 1 LEADER]: Becoming Leader. State: Replica: f06a395314b54e85a5ae33591c419e48, State: Running, Role: LEADER
I20260806 01:17:28.721678 17000 consensus_queue.cc:237] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } } peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } }
I20260806 01:17:28.727800 16322 catalog_manager.cc:5495] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 reported cstate change: term changed from 0 to 1, leader changed from <none> to f06a395314b54e85a5ae33591c419e48 (127.25.236.130). New cstate: current_term: 1 leader_uuid: "f06a395314b54e85a5ae33591c419e48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06a395314b54e85a5ae33591c419e48" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 41607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 } health_report { overall_health: UNKNOWN } } }
W20260806 01:17:28.750511 16963 tablet.cc:2369] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:17:28.756366 16980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.748968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48468 (local address 127.25.236.130:41607)
0806 01:17:28.749345 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:28.749357 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:28.749383 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:17:28.749647 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:28.749651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:28.749735 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:28.749876 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:17:28.750592 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.751721 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.753787 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.754406 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.754553 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.755101 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:17:28.755618 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:17:28.755778 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:28.755884 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 01:17:28.755968 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:17:28.765125 16515 raft_consensus.cc:1261] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98 [term 1 FOLLOWER]: Refusing update from remote peer f06a395314b54e85a5ae33591c419e48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:17:28.765239 16902 raft_consensus.cc:1261] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 1 FOLLOWER]: Refusing update from remote peer f06a395314b54e85a5ae33591c419e48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 01:17:28.765435 16782 tablet.cc:2369] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:17:28.766412 17007 consensus_queue.cc:1035] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6b160dbc174478a96c0f7c5cc342e98" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 46883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:17:28.766801 17000 consensus_queue.cc:1035] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:17:28.770586 16589 tablet.cc:2369] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:17:28.779119 17012 mvcc.cc:205] Tried to move back new op lower bound from 7315370183731920896 to 7315370183570599936. Current Snapshot: MvccSnapshot[applied={T|T < 7315370183731920896}]
I20260806 01:17:28.794575 16322 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:17:28.794888 16322 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:17:28.798874 16322 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Sending DeleteTablet for 3 replicas of tablet 5912af177c2d42728f702f34d4df6363
I20260806 01:17:28.800201 16873 tablet_service.cc:1514] Processing DeleteTablet for tablet 5912af177c2d42728f702f34d4df6363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:17:28 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:48848
I20260806 01:17:28.800227 16687 tablet_service.cc:1514] Processing DeleteTablet for tablet 5912af177c2d42728f702f34d4df6363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:17:28 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:48462
I20260806 01:17:28.800197 16492 tablet_service.cc:1514] Processing DeleteTablet for tablet 5912af177c2d42728f702f34d4df6363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:17:28 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:57872
I20260806 01:17:28.803130 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 16376
I20260806 01:17:28.813144 17023 tablet_replica.cc:330] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98: stopping tablet replica
I20260806 01:17:28.813972 17023 raft_consensus.cc:2229] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:17:28.814638 17023 raft_consensus.cc:2258] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:17:28.815662 17022 tablet_replica.cc:330] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48: stopping tablet replica
I20260806 01:17:28.816111 17022 raft_consensus.cc:2229] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:17:28.816816 17024 tablet_replica.cc:330] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: stopping tablet replica
I20260806 01:17:28.817219 17024 raft_consensus.cc:2229] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:17:28.817411 17022 raft_consensus.cc:2258] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:17:28.817734 17024 raft_consensus.cc:2258] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:17:28.819519 17023 ts_tablet_manager.cc:1905] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:17:28.819828 17022 ts_tablet_manager.cc:1905] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:17:28.822722 17024 ts_tablet_manager.cc:1905] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:17:28.827616 17022 ts_tablet_manager.cc:1918] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:17:28.827615 17023 ts_tablet_manager.cc:1918] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:17:28.827806 17022 log.cc:1196] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/5912af177c2d42728f702f34d4df6363
I20260806 01:17:28.828012 17023 log.cc:1196] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/5912af177c2d42728f702f34d4df6363
I20260806 01:17:28.828523 17022 ts_tablet_manager.cc:1939] T 5912af177c2d42728f702f34d4df6363 P f06a395314b54e85a5ae33591c419e48: Deleting consensus metadata
I20260806 01:17:28.828823 17023 ts_tablet_manager.cc:1939] T 5912af177c2d42728f702f34d4df6363 P b6b160dbc174478a96c0f7c5cc342e98: Deleting consensus metadata
I20260806 01:17:28.829092 16985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.805279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57888 (local address 127.25.236.129:46883)
0806 01:17:28.805566 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:28.805575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:28.805603 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:17:28.805679 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:28.805688 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:28.805809 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:28.805941 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:28.808581 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.810262 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.815685 (+ 5423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.816372 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.816501 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.821252 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:28.821269 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:28.821285 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:28.821328 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:28.825586 (+ 4258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:28.826544 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:28.826553 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:28.826557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:28.826734 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:28.827093 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:28.827098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:28.827100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:28.827610 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:28.827957 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:28.828269 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 01:17:28.828600 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260806 01:17:28.829988 16580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:28.831333 16302 catalog_manager.cc:4841] TS f06a395314b54e85a5ae33591c419e48 (127.25.236.130:41607): tablet 5912af177c2d42728f702f34d4df6363 (table test-table [id=e4a7c79700824ed980ba5cb664bd8b89]) successfully deleted
I20260806 01:17:28.833209 17024 ts_tablet_manager.cc:1918] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:17:28.833444 17024 log.cc:1196] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/5912af177c2d42728f702f34d4df6363
I20260806 01:17:28.833971 17024 ts_tablet_manager.cc:1939] T 5912af177c2d42728f702f34d4df6363 P 7ff69b6c87cc439c83f5a5a4dbd6f5ef: Deleting consensus metadata
I20260806 01:17:28.836615 16301 catalog_manager.cc:4841] TS 7ff69b6c87cc439c83f5a5a4dbd6f5ef (127.25.236.131:43555): tablet 5912af177c2d42728f702f34d4df6363 (table test-table [id=e4a7c79700824ed980ba5cb664bd8b89]) successfully deleted
I20260806 01:17:28.927021 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 16376
I20260806 01:17:28.927321 16303 catalog_manager.cc:4841] TS b6b160dbc174478a96c0f7c5cc342e98 (127.25.236.129:46883): tablet 5912af177c2d42728f702f34d4df6363 (table test-table [id=e4a7c79700824ed980ba5cb664bd8b89]) successfully deleted
I20260806 01:17:28.963253 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 16593
I20260806 01:17:28.981310 16980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:28.964062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48472 (local address 127.25.236.130:41607)
0806 01:17:28.964261 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:28.964267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:28.964394 (+ 127us) server_negotiation.cc:407] Connection header received
0806 01:17:28.964574 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:28.964580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:28.964663 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:28.964810 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:28.966695 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.967880 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.969908 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:28.970735 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:28.970904 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:28.974895 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:28.974914 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:28.974934 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:28.974980 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:28.978385 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:28.979228 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:28.979235 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:28.979237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:28.979423 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:28.979860 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:28.979863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:28.979865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:28.980221 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:28.980474 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:28.980808 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 01:17:28.980974 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:17:28.982023 16776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:29.101495 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 16593
I20260806 01:17:29.133584 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 16784
I20260806 01:17:29.151690 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:29.134632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48856 (local address 127.25.236.131:43555)
0806 01:17:29.134858 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:29.134866 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:29.134887 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:17:29.134957 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:29.134960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:29.135022 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:29.135277 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:29.136660 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:29.137405 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:29.138781 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:29.139196 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:29.139335 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:29.145946 (+ 6611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:29.145962 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:29.145978 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:29.146019 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:29.148763 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:29.149480 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:29.149487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:29.149489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:29.149637 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:29.150025 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:29.150029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:29.150031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:29.150439 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:29.150740 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:29.151109 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 01:17:29.151333 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 01:17:29.152514 16956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:29.243562 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 16784
I20260806 01:17:29.273944 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 16278
I20260806 01:17:29.275406 16363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:29.390054 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 16278
I20260806 01:17:29.421912 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:34371
--webserver_interface=127.25.236.190
--webserver_port=33505
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:34371
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:29.801014 17029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:29.801306 17029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:29.801396 17029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:29.808497 17029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:17:29.808629 17029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:29.808725 17029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:29.808835 17029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:17:29.808915 17029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:17:29.818303 17029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:34371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:34371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=33505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:29.819504 17029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:29.820757 17029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:29.832825 17036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:29.833072 17039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:29.833041 17035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:29.834091 17029 server_base.cc:1034] running on GCE node
I20260806 01:17:29.835072 17029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:29.836651 17029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:29.837888 17029 hybrid_clock.cc:648] HybridClock initialized: now 1785979049837848 us; error 31 us; skew 500 ppm
I20260806 01:17:29.838369 17029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:29 dist-test-slave-7nm7 krb5kdc[15471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979049, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:29.842041 17029 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:17:29.843791 17029 webserver.cc:460] Webserver started at http://127.25.236.190:33505/ using document root <none> and password file <none>
I20260806 01:17:29.844528 17029 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:29.844688 17029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:29.850960 17029 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 01:17:29.855147 17047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:29.857034 17029 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:17:29.857292 17029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "bb58a156f4ed4e1a85ce726195236e9d"
format_stamp: "Formatted at 2026-08-06 01:17:23 on dist-test-slave-7nm7"
server_key: "bbf78cbb4f5627395409ed06623e18da"
server_key_iv: "9073e950fdf61cf77c9c0460a096048f"
server_key_version: "encryptionkey@0"
I20260806 01:17:29.858446 17029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:29.878683 17029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:29.884127 17029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:29.884439 17029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:29.907272 17029 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:34371
I20260806 01:17:29.907292 17118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:34371 every 8 connection(s)
I20260806 01:17:29.909129 17029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:17:29.914183 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 17029
I20260806 01:17:29.915097 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:46883
--local_ip_for_outbound_sockets=127.25.236.129
--tserver_master_addrs=127.25.236.190:34371
--webserver_port=43949
--webserver_interface=127.25.236.129
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:17:29.917172 17119 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:17:29.921607 17119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Bootstrap starting.
I20260806 01:17:29.949380 17119 log.cc:824] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:29.966670 17119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:17:29.967239 17119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d: Bootstrap complete.
I20260806 01:17:29.978114 17119 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } }
I20260806 01:17:29.979147 17119 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb58a156f4ed4e1a85ce726195236e9d, State: Initialized, Role: FOLLOWER
I20260806 01:17:29.979801 17119 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } }
I20260806 01:17:29.980057 17119 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:17:29.980201 17119 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:17:29.980402 17119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 2 FOLLOWER]: Advancing to term 3
I20260806 01:17:29.984673 17119 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } }
I20260806 01:17:29.985227 17119 leader_election.cc:302] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb58a156f4ed4e1a85ce726195236e9d; no voters:
I20260806 01:17:29.986104 17119 leader_election.cc:288] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 01:17:29.986267 17124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 3 FOLLOWER]: Leader election won for term 3
I20260806 01:17:29.987699 17124 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [term 3 LEADER]: Becoming Leader. State: Replica: bb58a156f4ed4e1a85ce726195236e9d, State: Running, Role: LEADER
I20260806 01:17:29.988315 17124 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } }
I20260806 01:17:29.988646 17119 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:17:29.991353 17125 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "bb58a156f4ed4e1a85ce726195236e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } } }
I20260806 01:17:29.991734 17125 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:29.992197 17128 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb58a156f4ed4e1a85ce726195236e9d. Latest consensus state: current_term: 3 leader_uuid: "bb58a156f4ed4e1a85ce726195236e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb58a156f4ed4e1a85ce726195236e9d" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 34371 } } }
I20260806 01:17:29.992448 17128 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb58a156f4ed4e1a85ce726195236e9d [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:30.003189 17133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:17:30.014770 17133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:17:30.016793 17133 catalog_manager.cc:1259] Loaded cluster ID: e1a0704b68344fae913117409674e8aa
I20260806 01:17:30.016988 17133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 01:17:30.019822 17133 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3d88c219ad google::LogMessage::Fail() at ??:0
@ 0x7f3d88c2488a google::LogMessage::SendToLog() at ??:0
@ 0x7f3d88c214de google::LogMessage::Flush() at ??:0
@ 0x7f3d88c22a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3d9e654512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f3d9e651976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3d9e6d9610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f3d9e6d9439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3d8b86d345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3d8b87091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3d8b870799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3d8b84beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3d8eb256db start_thread at ??:0
@ 0x7f3d87abb61f clone at ??:0
W20260806 01:17:30.333899 17122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:30.334229 17122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:30.334424 17122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:30.341684 17122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:30.341852 17122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:30.342044 17122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:17:30.353129 17122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:46883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=43949
--tserver_master_addrs=127.25.236.190:34371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:30.354805 17122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:30.356247 17122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:30.370134 17186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:30.370316 17184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:30.370285 17183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:30.371250 17122 server_base.cc:1034] running on GCE node
I20260806 01:17:30.371887 17122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:30.372769 17122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:30.373978 17122 hybrid_clock.cc:648] HybridClock initialized: now 1785979050373919 us; error 54 us; skew 500 ppm
I20260806 01:17:30.374365 17122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:30 dist-test-slave-7nm7 krb5kdc[15471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979050, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:30.377483 17122 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:17:30.379031 17122 webserver.cc:460] Webserver started at http://127.25.236.129:43949/ using document root <none> and password file <none>
I20260806 01:17:30.379763 17122 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:30.379913 17122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:30.385651 17122 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 01:17:30.388410 17194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:30.389760 17122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:17:30.389947 17122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "b6b160dbc174478a96c0f7c5cc342e98"
format_stamp: "Formatted at 2026-08-06 01:17:24 on dist-test-slave-7nm7"
server_key: "31b0883aa94446e1e2447469283e8607"
server_key_iv: "81cf14ad9375f31a7721ac03df5db2db"
server_key_version: "encryptionkey@0"
I20260806 01:17:30.391132 17122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:30.412837 17122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:30.417579 17122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:30.417896 17122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:30.419063 17122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:30.420619 17122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:30.420774 17122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:30.420930 17122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:30.421027 17122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:30.465165 17122 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:46883
I20260806 01:17:30.465225 17366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:46883 every 8 connection(s)
I20260806 01:17:30.466737 17122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:17:30.471575 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:30.469732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:40967)
0806 01:17:30.470540 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:30.470758 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":304,"threads_started":1}
W20260806 01:17:30.472618 17369 heartbeater.cc:644] Failed to heartbeat to 127.25.236.190:34371 (0 consecutive failures): Network error: Failed to ping master at 127.25.236.190:34371: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
I20260806 01:17:30.473600 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:30.473216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:42399)
0806 01:17:30.473391 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:30.473482 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 01:17:30.474606 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:30.474195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:38609)
0806 01:17:30.474412 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:30.474488 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260806 01:17:30.474831 17369 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 01:17:30.475260 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 17122
I20260806 01:17:30.475895 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:41607
--local_ip_for_outbound_sockets=127.25.236.130
--tserver_master_addrs=127.25.236.190:34371
--webserver_port=45817
--webserver_interface=127.25.236.130
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:30.845026 17372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:30.845348 17372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:30.845453 17372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:30.853926 17372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:30.854063 17372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:30.854367 17372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:17:30.865410 17372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:41607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=45817
--tserver_master_addrs=127.25.236.190:34371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:30.867285 17372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:30.868677 17372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:30.883883 17380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:30.883760 17383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:30.883788 17385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:30.885332 17372 server_base.cc:1034] running on GCE node
I20260806 01:17:30.885993 17372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:30.887075 17372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:30.888319 17372 hybrid_clock.cc:648] HybridClock initialized: now 1785979050888253 us; error 61 us; skew 500 ppm
I20260806 01:17:30.888696 17372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:30 dist-test-slave-7nm7 krb5kdc[15471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979050, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:30.892802 17372 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:17:30.894513 17372 webserver.cc:460] Webserver started at http://127.25.236.130:45817/ using document root <none> and password file <none>
I20260806 01:17:30.895325 17372 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:30.895488 17372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:30.901342 17372 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 01:17:30.904706 17395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:30.906467 17372 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 01:17:30.906656 17372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "f06a395314b54e85a5ae33591c419e48"
format_stamp: "Formatted at 2026-08-06 01:17:24 on dist-test-slave-7nm7"
server_key: "2647ede744b85f4e3e789eab15313028"
server_key_iv: "6898e73898df6e9d94a822fbcd273e75"
server_key_version: "encryptionkey@0"
I20260806 01:17:30.908008 17372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:30.923079 17372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:30.929173 17372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:30.929430 17372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:30.930711 17372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:30.932216 17372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:30.932320 17372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:30.932436 17372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:30.932528 17372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:30.969264 17372 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:41607
I20260806 01:17:30.969527 17560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:41607 every 8 connection(s)
I20260806 01:17:30.970898 17372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:17:30.976960 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:30.974912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:47043)
0806 01:17:30.975847 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:30.976075 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":362,"threads_started":1}
W20260806 01:17:30.977741 17561 heartbeater.cc:644] Failed to heartbeat to 127.25.236.190:34371 (0 consecutive failures): Network error: Failed to ping master at 127.25.236.190:34371: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
I20260806 01:17:30.978935 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:30.978586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:43101)
0806 01:17:30.978753 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:30.978830 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 01:17:30.979244 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 17372
I20260806 01:17:30.980016 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:30.979650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:35159)
0806 01:17:30.979799 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:30.979892 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 01:17:30.980003 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:43555
--local_ip_for_outbound_sockets=127.25.236.131
--tserver_master_addrs=127.25.236.190:34371
--webserver_port=35157
--webserver_interface=127.25.236.131
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:30.980450 17561 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:17:31.360188 17567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:31.360496 17567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:31.360661 17567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:31.369031 17567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:31.369402 17567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:31.369586 17567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:17:31.383315 17567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:43555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=35157
--tserver_master_addrs=127.25.236.190:34371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:31.384727 17567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:31.386407 17567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:31.400750 17575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:31.400695 17574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:31.401202 17579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:31.401870 17567 server_base.cc:1034] running on GCE node
I20260806 01:17:31.402700 17567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:31.403935 17567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:31.405205 17567 hybrid_clock.cc:648] HybridClock initialized: now 1785979051405152 us; error 44 us; skew 500 ppm
I20260806 01:17:31.405551 17567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:31 dist-test-slave-7nm7 krb5kdc[15471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979051, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:31.409093 17567 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:17:31.411350 17567 webserver.cc:460] Webserver started at http://127.25.236.131:35157/ using document root <none> and password file <none>
I20260806 01:17:31.412132 17567 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:31.412295 17567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:31.418881 17567 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 01:17:31.421718 17593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:31.423271 17567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:17:31.423461 17567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "7ff69b6c87cc439c83f5a5a4dbd6f5ef"
format_stamp: "Formatted at 2026-08-06 01:17:25 on dist-test-slave-7nm7"
server_key: "2f266af31248d6d3ee68979c7c2e58c7"
server_key_iv: "4ed8c76c43e3952509e828d223a2e184"
server_key_version: "encryptionkey@0"
I20260806 01:17:31.424676 17567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:31.444964 17567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:31.450954 17567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:31.451344 17567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:31.452903 17567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:31.454618 17567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:31.454782 17567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:31.454917 17567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:31.455009 17567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:31.476701 17677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:31.475530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:40619)
0806 01:17:31.476132 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:31.476344 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":255,"threads_started":1}
I20260806 01:17:31.496016 17567 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:43555
I20260806 01:17:31.496184 17760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:43555 every 8 connection(s)
I20260806 01:17:31.497738 17567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:17:31.498458 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 17567
I20260806 01:17:31.502784 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:31.500880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:35611)
0806 01:17:31.501704 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:31.501921 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":387,"threads_started":1}
W20260806 01:17:31.503569 17763 heartbeater.cc:644] Failed to heartbeat to 127.25.236.190:34371 (0 consecutive failures): Network error: Failed to ping master at 127.25.236.190:34371: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
I20260806 01:17:31.504383 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:31.504041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:32889)
0806 01:17:31.504196 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:31.504265 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 01:17:31.505381 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:31.504971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:44103)
0806 01:17:31.505193 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:31.505273 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260806 01:17:31.505640 17763 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 01:17:31.982131 17768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:31.981183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:59531)
0806 01:17:31.981749 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:31.981869 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":312,"threads_started":1}
I20260806 01:17:32.479656 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:32.478429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:53569)
0806 01:17:32.479193 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:32.479309 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":424,"threads_started":1}
I20260806 01:17:32.507405 17771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:32.506196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:57563)
0806 01:17:32.506965 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:32.507096 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":458,"threads_started":1}
I20260806 01:17:32.984390 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:32.983327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:36253)
0806 01:17:32.984007 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:32.984187 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":308,"threads_started":1}
I20260806 01:17:33.481918 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:33.480899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:47865)
0806 01:17:33.481538 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:33.481662 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":339,"threads_started":1}
I20260806 01:17:33.509117 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:33.508423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:42261)
0806 01:17:33.508871 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:33.508949 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":220,"threads_started":1}
I20260806 01:17:33.986173 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:33.985459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:59211)
0806 01:17:33.985906 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:33.985988 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":251,"threads_started":1}
I20260806 01:17:34.483911 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:34.483161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:53357)
0806 01:17:34.483647 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:34.483737 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":244,"threads_started":1}
I20260806 01:17:34.510839 17777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:34.510048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:57637)
0806 01:17:34.510588 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:34.510670 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":326,"threads_started":1}
I20260806 01:17:34.988005 17778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:34.987195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:33777)
0806 01:17:34.987763 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:34.987841 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":302,"threads_started":1}
I20260806 01:17:35.485944 17780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:35.484944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:35933)
0806 01:17:35.485674 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:35.485762 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":502,"threads_started":1}
I20260806 01:17:35.512776 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:35.511766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:44737)
0806 01:17:35.512425 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:35.512547 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":405,"threads_started":1}
I20260806 01:17:35.990338 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:35.989333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:56255)
0806 01:17:35.989838 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:35.990067 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":263,"threads_started":1}
I20260806 01:17:36.488322 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:36.487393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:36519)
0806 01:17:36.487992 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:36.488115 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":301,"threads_started":1}
I20260806 01:17:36.514811 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:36.513874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:36667)
0806 01:17:36.514534 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:36.514623 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":272,"threads_started":1}
I20260806 01:17:36.991995 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:36.991269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:52663)
0806 01:17:36.991742 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:36.991838 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":238,"threads_started":1}
I20260806 01:17:37.490780 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:37.489814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:37049)
0806 01:17:37.490465 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:37.490559 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":289,"threads_started":1}
I20260806 01:17:37.516544 17790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:37.515821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:48657)
0806 01:17:37.516286 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:37.516377 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":247,"threads_started":1}
I20260806 01:17:37.993875 17791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:37.992995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:33941)
0806 01:17:37.993552 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:37.993632 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":238,"threads_started":1}
I20260806 01:17:38.492911 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:38.491828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:39449)
0806 01:17:38.492561 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:38.492689 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":422,"threads_started":1}
I20260806 01:17:38.518395 17793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:38.517515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:37893)
0806 01:17:38.518007 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:38.518092 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":236,"threads_started":1}
I20260806 01:17:38.995903 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:38.994910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:49727)
0806 01:17:38.995519 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:38.995611 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":290,"threads_started":1}
I20260806 01:17:39.494777 17796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:39.493884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:39489)
0806 01:17:39.494504 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:39.494599 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":393,"threads_started":1}
I20260806 01:17:39.520051 17798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:39.519312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:50537)
0806 01:17:39.519795 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:39.519886 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":247,"threads_started":1}
I20260806 01:17:39.998171 17799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:39.997168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:58207)
0806 01:17:39.997876 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:39.997968 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":423,"threads_started":1}
I20260806 01:17:40.496814 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:40.495879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:47131)
0806 01:17:40.496549 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:40.496652 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":431,"threads_started":1}
I20260806 01:17:40.522854 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:40.521657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:56633)
0806 01:17:40.522506 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:40.522625 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":325,"threads_started":1}
I20260806 01:17:41.000294 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:40.999537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:60995)
0806 01:17:41.000018 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:41.000111 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":270,"threads_started":1}
I20260806 01:17:41.499150 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:41.497925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:33103)
0806 01:17:41.498713 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:41.498839 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":554,"threads_started":1}
I20260806 01:17:41.525110 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:41.524346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:45079)
0806 01:17:41.524834 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:41.524929 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":253,"threads_started":1}
I20260806 01:17:42.002105 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:42.001372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:52457)
0806 01:17:42.001832 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:42.001914 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":234,"threads_started":1}
I20260806 01:17:42.501646 17812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:42.500715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:41639)
0806 01:17:42.501368 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:42.501471 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":335,"threads_started":1}
I20260806 01:17:42.527709 17813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:42.526520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:40305)
0806 01:17:42.527326 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:42.527463 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":456,"threads_started":1}
I20260806 01:17:43.004211 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:43.003164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:33921)
0806 01:17:43.003845 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:43.003994 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":348,"threads_started":1}
I20260806 01:17:43.503486 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:43.502644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:38693)
0806 01:17:43.503216 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:43.503305 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":296,"threads_started":1}
I20260806 01:17:43.529757 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:43.528921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:37417)
0806 01:17:43.529498 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:43.529594 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":321,"threads_started":1}
I20260806 01:17:44.006300 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:44.005395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:45415)
0806 01:17:44.005889 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:44.006031 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":256,"threads_started":1}
I20260806 01:17:44.505651 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:44.504748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:60445)
0806 01:17:44.505377 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:44.505467 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":402,"threads_started":1}
I20260806 01:17:44.531482 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:44.530702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:35383)
0806 01:17:44.531195 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:44.531286 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":231,"threads_started":1}
I20260806 01:17:45.007930 17821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:45.007234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:50917)
0806 01:17:45.007651 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:45.007732 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":212,"threads_started":1}
I20260806 01:17:45.507942 17822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:45.506956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:40995)
0806 01:17:45.507655 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:45.507754 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":349,"threads_started":1}
I20260806 01:17:45.533331 17823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:45.532546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:48575)
0806 01:17:45.533024 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:45.533147 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":238,"threads_started":1}
I20260806 01:17:46.009974 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:46.009187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.130:44345)
0806 01:17:46.009700 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:46.009800 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":272,"threads_started":1}
I20260806 01:17:46.504877 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 17122
I20260806 01:17:46.509835 17828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:46.508894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.129:44479)
0806 01:17:46.509407 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:46.509561 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":256,"threads_started":1}
I20260806 01:17:46.526743 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:46.506110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50612 (local address 127.25.236.129:46883)
0806 01:17:46.508573 (+ 2463us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:46.508590 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:46.508659 (+ 69us) server_negotiation.cc:407] Connection header received
0806 01:17:46.510956 (+ 2297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:46.510966 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:46.511412 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:46.511763 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:46.513474 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:46.514826 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:46.516236 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:46.516923 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:46.517065 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:46.519872 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:46.519880 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:46.519894 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:46.519936 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:46.523264 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:46.523934 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:46.523941 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:46.523947 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:46.524093 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:46.524491 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:46.524494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:46.524496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:46.525024 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:46.525408 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:46.525887 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 01:17:46.526425 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2257,"thread_start_us":2071,"threads_started":1}
I20260806 01:17:46.528250 17359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:46.535358 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:46.534575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:34371 (local address 127.25.236.131:52581)
0806 01:17:46.535049 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:46.535165 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":242,"threads_started":1}
I20260806 01:17:46.619390 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 17122
I20260806 01:17:46.652045 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 17372
I20260806 01:17:46.674757 17834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:46.653338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41816 (local address 127.25.236.130:41607)
0806 01:17:46.653967 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:46.653998 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:46.654039 (+ 41us) server_negotiation.cc:407] Connection header received
0806 01:17:46.656069 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:46.656080 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:46.656528 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:46.656907 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:46.658868 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:46.660632 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:46.662553 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:46.663319 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:46.663553 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:46.667422 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:46.667431 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:46.667450 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:46.667495 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:46.670806 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:46.671931 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:46.671938 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:46.671945 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:46.672145 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:46.672735 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:46.672739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:46.672742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:46.673322 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:46.673568 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:46.674011 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 01:17:46.674453 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":307,"threads_started":1}
I20260806 01:17:46.676208 17551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:46.772187 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 17372
I20260806 01:17:46.806198 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 17567
I20260806 01:17:46.829495 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:46.807829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48224 (local address 127.25.236.131:43555)
0806 01:17:46.808541 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:46.808556 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:46.808612 (+ 56us) server_negotiation.cc:407] Connection header received
0806 01:17:46.810726 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:46.810744 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:46.811362 (+ 618us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:46.811736 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:46.813353 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:46.814687 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:46.816027 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:46.816718 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:46.816930 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:46.820192 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:46.820200 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:46.820217 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:46.820262 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:46.824401 (+ 4139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:46.825920 (+ 1519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:46.825930 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:46.825940 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:46.826224 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:46.827064 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:46.827075 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:46.827077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:46.827817 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:46.828181 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:46.828683 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 01:17:46.829144 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":361,"threads_started":1}
I20260806 01:17:46.831024 17754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:46.923238 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 17567
2026-08-06T01:17:46Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25652 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:17:47 dist-test-slave-7nm7 krb5kdc[17842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:17:47 dist-test-slave-7nm7 krb5kdc[17842](info): set up 2 sockets
Aug 06 01:17:47 dist-test-slave-7nm7 krb5kdc[17842](info): commencing operation
krb5kdc: starting...
W20260806 01:17:48.312232 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.336s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:17:48 dist-test-slave-7nm7 krb5kdc[17842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:17:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:17:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:17:48.645804 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:35573
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:35573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:49.018191 17864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:49.018635 17864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:49.018795 17864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:49.026731 17864 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 01:17:49.026889 17864 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 01:17:49.026990 17864 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 01:17:49.027091 17864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:17:49.027314 17864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:49.027452 17864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:49.027544 17864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:17:49.027630 17864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:17:49.038081 17864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:35573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:35573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:49.039491 17864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:49.040943 17864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:49.054023 17871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:49.053956 17879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:49.053915 17872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:49.054322 17864 server_base.cc:1034] running on GCE node
I20260806 01:17:49.055946 17864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:49.057588 17864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:49.058888 17864 hybrid_clock.cc:648] HybridClock initialized: now 1785979069058818 us; error 48 us; skew 500 ppm
I20260806 01:17:49.059386 17864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:49 dist-test-slave-7nm7 krb5kdc[17842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979069, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:49.066489 17864 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:17:49.068037 17864 webserver.cc:460] Webserver started at http://127.25.236.190:41691/ using document root <none> and password file <none>
I20260806 01:17:49.068758 17864 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:49.068856 17864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:49.069119 17864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:17:49.072218 17864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "28b759deabf24687960b360cdebb0ecc"
format_stamp: "Formatted at 2026-08-06 01:17:49 on dist-test-slave-7nm7"
server_key: "523a7de0a792a90de83d98afecf17bf9"
server_key_iv: "970d187d5fe9e955cf31b19cdb889bf7"
server_key_version: "encryptionkey@0"
I20260806 01:17:49.072983 17864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "28b759deabf24687960b360cdebb0ecc"
format_stamp: "Formatted at 2026-08-06 01:17:49 on dist-test-slave-7nm7"
server_key: "523a7de0a792a90de83d98afecf17bf9"
server_key_iv: "970d187d5fe9e955cf31b19cdb889bf7"
server_key_version: "encryptionkey@0"
I20260806 01:17:49.078331 17864 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 01:17:49.081785 17890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:49.083210 17864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:17:49.083393 17864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "28b759deabf24687960b360cdebb0ecc"
format_stamp: "Formatted at 2026-08-06 01:17:49 on dist-test-slave-7nm7"
server_key: "523a7de0a792a90de83d98afecf17bf9"
server_key_iv: "970d187d5fe9e955cf31b19cdb889bf7"
server_key_version: "encryptionkey@0"
I20260806 01:17:49.083619 17864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:49.103420 17864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:49.108455 17864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:49.108796 17864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:49.131517 17953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:35573 every 8 connection(s)
I20260806 01:17:49.131731 17864 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:35573
I20260806 01:17:49.133404 17864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:17:49.137759 17956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:49.143203 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 17864
I20260806 01:17:49.143467 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:17:49.144184 26546 external_mini_cluster.cc:1428] Setting key 781057ca8db88327c217b285c6db51d3
I20260806 01:17:49.149942 17956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Bootstrap starting.
I20260806 01:17:49.155441 17956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:49.156916 17956 log.cc:824] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:49.161005 17956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: No bootstrap required, opened a new log
Aug 06 01:17:49 dist-test-slave-7nm7 krb5kdc[17842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:49.170202 17956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } }
I20260806 01:17:49.170742 17956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:49.170882 17956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b759deabf24687960b360cdebb0ecc, State: Initialized, Role: FOLLOWER
I20260806 01:17:49.171656 17956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } }
I20260806 01:17:49.171936 17956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:17:49.172102 17956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:17:49.172297 17956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:49.172542 17961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:49.146451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46542 (local address 127.25.236.190:35573)
0806 01:17:49.148001 (+ 1550us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:49.148022 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:49.148103 (+ 81us) server_negotiation.cc:407] Connection header received
0806 01:17:49.150616 (+ 2513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:49.150654 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:49.151212 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:49.151672 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:49.154057 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:49.155575 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:49.157988 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:49.159052 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:49.159199 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:49.164137 (+ 4938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:49.164146 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:49.164163 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:49.164210 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:49.167955 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:49.168792 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:49.168799 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:49.168809 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:49.168987 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:49.169403 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:49.169410 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:49.169418 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:49.170008 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:49.170359 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:49.170875 (+ 516us) server_negotiation.cc:299] Negotiation successful
0806 01:17:49.171343 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1122,"thread_start_us":638,"threads_started":1}
I20260806 01:17:49.175443 17956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } }
I20260806 01:17:49.176011 17956 leader_election.cc:302] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28b759deabf24687960b360cdebb0ecc; no voters:
I20260806 01:17:49.177014 17956 leader_election.cc:288] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:17:49.177121 17964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:17:49.177911 17964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 1 LEADER]: Becoming Leader. State: Replica: 28b759deabf24687960b360cdebb0ecc, State: Running, Role: LEADER
I20260806 01:17:49.178737 17964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } }
I20260806 01:17:49.179553 17956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:17:49.182247 17965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28b759deabf24687960b360cdebb0ecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } } }
I20260806 01:17:49.182652 17965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:49.182219 17968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28b759deabf24687960b360cdebb0ecc. Latest consensus state: current_term: 1 leader_uuid: "28b759deabf24687960b360cdebb0ecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } } }
I20260806 01:17:49.183199 17968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: This master's current role is: LEADER
I20260806 01:17:49.185403 17982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:17:49.191244 17982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:17:49.202174 17982 catalog_manager.cc:1347] Generated new cluster ID: eaffb1d7f802451b923b56bb73c3c423
I20260806 01:17:49.202327 17982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:17:49.226966 17982 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:17:49.228816 17982 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:17:49.241688 17982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Generated new TSK 0
I20260806 01:17:49.242759 17982 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:17:49.326666 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:35573
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:49.702337 18000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:49.702641 18000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:49.702793 18000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:49.710184 18000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:49.710391 18000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:49.710600 18000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:17:49.720427 18000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:35573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:49.722084 18000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:49.723511 18000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:49.740341 18006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:49.740298 18007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:49.740330 18010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:49.741452 18000 server_base.cc:1034] running on GCE node
I20260806 01:17:49.742307 18000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:49.743387 18000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:49.744599 18000 hybrid_clock.cc:648] HybridClock initialized: now 1785979069744556 us; error 40 us; skew 500 ppm
I20260806 01:17:49.745002 18000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:49 dist-test-slave-7nm7 krb5kdc[17842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979069, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:49.748399 18000 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:17:49.749991 18000 webserver.cc:460] Webserver started at http://127.25.236.129:43869/ using document root <none> and password file <none>
I20260806 01:17:49.750769 18000 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:49.750928 18000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:49.751259 18000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:17:49.754580 18000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "e711d05b3eda42a1821405b09d4fa7bb"
format_stamp: "Formatted at 2026-08-06 01:17:49 on dist-test-slave-7nm7"
server_key: "3b2089675dce3be8ff4e72b4442b14e9"
server_key_iv: "78e283c4a2307d6b7bbb654c823478a7"
server_key_version: "encryptionkey@0"
I20260806 01:17:49.755426 18000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "e711d05b3eda42a1821405b09d4fa7bb"
format_stamp: "Formatted at 2026-08-06 01:17:49 on dist-test-slave-7nm7"
server_key: "3b2089675dce3be8ff4e72b4442b14e9"
server_key_iv: "78e283c4a2307d6b7bbb654c823478a7"
server_key_version: "encryptionkey@0"
I20260806 01:17:49.761348 18000 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 01:17:49.765174 18017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:49.766626 18000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:17:49.766834 18000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "e711d05b3eda42a1821405b09d4fa7bb"
format_stamp: "Formatted at 2026-08-06 01:17:49 on dist-test-slave-7nm7"
server_key: "3b2089675dce3be8ff4e72b4442b14e9"
server_key_iv: "78e283c4a2307d6b7bbb654c823478a7"
server_key_version: "encryptionkey@0"
I20260806 01:17:49.767062 18000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:49.826046 18000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:49.831334 18000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:49.831712 18000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:49.832907 18000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:49.834421 18000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:49.834537 18000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:49.834702 18000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:49.834787 18000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:49.876256 18000 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:41553
I20260806 01:17:49.876555 18182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:41553 every 8 connection(s)
I20260806 01:17:49.877849 18000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:17:49.881863 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 18000
I20260806 01:17:49.882160 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:17:49.883337 26546 external_mini_cluster.cc:1428] Setting key 110aa34d77e411c2d564589e6e013ec3
Aug 06 01:17:49 dist-test-slave-7nm7 krb5kdc[17842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979069, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:49.904551 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:49.880820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:58667 (local address 127.25.236.190:35573)
0806 01:17:49.884447 (+ 3627us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:49.884455 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:49.884476 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:17:49.884677 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:49.884687 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:49.884767 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:49.884869 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:49.886847 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:49.887878 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:49.890252 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:49.890974 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:49.891141 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:49.896870 (+ 5729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:49.896879 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:49.896886 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:49.896933 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:49.900414 (+ 3481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:49.901618 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:49.901627 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:49.901634 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:49.901798 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:49.902341 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:49.902349 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:49.902356 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:49.902787 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:49.902952 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:49.903761 (+ 809us) server_negotiation.cc:299] Negotiation successful
0806 01:17:49.904030 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3477,"thread_start_us":3365,"threads_started":1}
I20260806 01:17:49.905210 18188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:49.881586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:58667)
0806 01:17:49.882480 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:49.882528 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:49.884431 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:49.885156 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:49.885168 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:49.885894 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:49.886655 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:49.886684 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:49.888072 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:49.888080 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:49.889998 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:49.890021 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:49.891223 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:49.891231 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:49.891566 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:49.892658 (+ 1092us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:49.892706 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:49.896599 (+ 3893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:49.900630 (+ 4031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:49.900643 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:49.900666 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:49.901426 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:49.901925 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:49.901928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:49.901930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:49.902127 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:49.902941 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:49.902947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:49.903323 (+ 376us) client_negotiation.cc:769] Sending connection context
0806 01:17:49.903579 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 01:17:49.904008 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":365,"threads_started":1}
I20260806 01:17:49.906981 18183 heartbeater.cc:347] Connected to a master server at 127.25.236.190:35573
I20260806 01:17:49.907644 18183 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:49.908840 18183 heartbeater.cc:511] Master 127.25.236.190:35573 requested a full tablet report, sending...
I20260806 01:17:49.911935 17909 ts_manager.cc:194] Registered new tserver with Master: e711d05b3eda42a1821405b09d4fa7bb (127.25.236.129:41553)
I20260806 01:17:49.913476 17909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:58667
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:17:49.956918 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:35573
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:50.325573 18191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:50.325906 18191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:50.326084 18191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:17:50.336083 18191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:50.336314 18191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:50.336642 18191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:17:50.348093 18191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:35573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:50.349419 18191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:50.350842 18191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:50.364548 18198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:50.364588 18201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:50.365293 18209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:50.366287 18191 server_base.cc:1034] running on GCE node
I20260806 01:17:50.366946 18191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:50.367859 18191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:50.369074 18191 hybrid_clock.cc:648] HybridClock initialized: now 1785979070369018 us; error 48 us; skew 500 ppm
I20260806 01:17:50.369536 18191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:50 dist-test-slave-7nm7 krb5kdc[17842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979070, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:50.373010 18191 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:17:50.374636 18191 webserver.cc:460] Webserver started at http://127.25.236.130:34109/ using document root <none> and password file <none>
I20260806 01:17:50.375341 18191 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:50.375487 18191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:50.375801 18191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:17:50.378970 18191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "b98987123d3e4758abbf3b5279a7d349"
format_stamp: "Formatted at 2026-08-06 01:17:50 on dist-test-slave-7nm7"
server_key: "614cb15279c875d08b76f639637d7f07"
server_key_iv: "81651d8e41f37bd5157a1d9974728562"
server_key_version: "encryptionkey@0"
I20260806 01:17:50.379858 18191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "b98987123d3e4758abbf3b5279a7d349"
format_stamp: "Formatted at 2026-08-06 01:17:50 on dist-test-slave-7nm7"
server_key: "614cb15279c875d08b76f639637d7f07"
server_key_iv: "81651d8e41f37bd5157a1d9974728562"
server_key_version: "encryptionkey@0"
I20260806 01:17:50.385385 18191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:17:50.389218 18220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:50.390690 18191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:17:50.390880 18191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "b98987123d3e4758abbf3b5279a7d349"
format_stamp: "Formatted at 2026-08-06 01:17:50 on dist-test-slave-7nm7"
server_key: "614cb15279c875d08b76f639637d7f07"
server_key_iv: "81651d8e41f37bd5157a1d9974728562"
server_key_version: "encryptionkey@0"
I20260806 01:17:50.391103 18191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:50.404651 18191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:50.409446 18191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:50.409801 18191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:50.410936 18191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:50.412384 18191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:50.412495 18191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:50.412647 18191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:50.412729 18191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:50.453045 18191 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:43759
I20260806 01:17:50.453085 18397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:43759 every 8 connection(s)
I20260806 01:17:50.454649 18191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:17:50.460609 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 18191
I20260806 01:17:50.460827 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:17:50.461303 26546 external_mini_cluster.cc:1428] Setting key 4b669b7853e25ffaa15cdc134957552d
Aug 06 01:17:50 dist-test-slave-7nm7 krb5kdc[17842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979070, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:50.481274 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:50.457733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:50863 (local address 127.25.236.190:35573)
0806 01:17:50.458178 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:50.458183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:50.459274 (+ 1091us) server_negotiation.cc:407] Connection header received
0806 01:17:50.461276 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:50.461282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:50.461376 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:50.461509 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:50.463469 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:50.464260 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:50.467090 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:50.467846 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:50.468036 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:50.473586 (+ 5550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:50.473592 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:50.473595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:50.473639 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:50.477545 (+ 3906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:50.478487 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:50.478494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:50.478503 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:50.478641 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:50.479100 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:50.479105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:50.479107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:50.479504 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:50.479682 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:50.480546 (+ 864us) server_negotiation.cc:299] Negotiation successful
0806 01:17:50.480863 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":213,"threads_started":1}
I20260806 01:17:50.481992 18403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:50.458369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:50863)
0806 01:17:50.459086 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:50.459133 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:50.460991 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:50.461795 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:50.461807 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:50.462571 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:50.463203 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:50.463231 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:50.464436 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:50.464445 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:50.466870 (+ 2425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:50.466896 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:50.468192 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:50.468202 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:50.468553 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:50.469421 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:50.469459 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:50.473272 (+ 3813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:50.477723 (+ 4451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:50.477735 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:50.477762 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:50.478354 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:50.478762 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:50.478767 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:50.478771 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:50.478971 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:50.479660 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:50.479667 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:50.480117 (+ 450us) client_negotiation.cc:769] Sending connection context
0806 01:17:50.480376 (+ 259us) client_negotiation.cc:240] Negotiation successful
0806 01:17:50.480788 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":286,"threads_started":1}
I20260806 01:17:50.484023 18398 heartbeater.cc:347] Connected to a master server at 127.25.236.190:35573
I20260806 01:17:50.484385 18398 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:50.485328 18398 heartbeater.cc:511] Master 127.25.236.190:35573 requested a full tablet report, sending...
I20260806 01:17:50.487447 17909 ts_manager.cc:194] Registered new tserver with Master: b98987123d3e4758abbf3b5279a7d349 (127.25.236.130:43759)
I20260806 01:17:50.488647 17909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:50863
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:17:50.537761 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:35573
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:17:50.908013 18407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:17:50.908375 18407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:17:50.908478 18407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 01:17:50.917174 18183 heartbeater.cc:503] Master 127.25.236.190:35573 was elected leader, sending a full tablet report...
W20260806 01:17:50.918301 18407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:17:50.918433 18407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:17:50.918648 18407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:17:50.929670 18407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:35573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:17:50.931178 18407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:17:50.932790 18407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:17:50.947314 18418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:50.947403 18415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:17:50.947360 18413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:17:50.948720 18407 server_base.cc:1034] running on GCE node
I20260806 01:17:50.949724 18407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:17:50.951738 18407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:17:50.953076 18407 hybrid_clock.cc:648] HybridClock initialized: now 1785979070952982 us; error 130 us; skew 500 ppm
I20260806 01:17:50.953532 18407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:17:50 dist-test-slave-7nm7 krb5kdc[17842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979070, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:17:50.957453 18407 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:17:50.959888 18407 webserver.cc:460] Webserver started at http://127.25.236.131:38785/ using document root <none> and password file <none>
I20260806 01:17:50.960655 18407 fs_manager.cc:359] Metadata directory not provided
I20260806 01:17:50.960772 18407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:17:50.961071 18407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:17:50.964377 18407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "e4ce8aa66b2b4b348b7bf5340446f02f"
format_stamp: "Formatted at 2026-08-06 01:17:50 on dist-test-slave-7nm7"
server_key: "bb813128446ec54169bd95e1a6fadc36"
server_key_iv: "2a24daa8a83d44e5a6974e9c509772f5"
server_key_version: "encryptionkey@0"
I20260806 01:17:50.965150 18407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "e4ce8aa66b2b4b348b7bf5340446f02f"
format_stamp: "Formatted at 2026-08-06 01:17:50 on dist-test-slave-7nm7"
server_key: "bb813128446ec54169bd95e1a6fadc36"
server_key_iv: "2a24daa8a83d44e5a6974e9c509772f5"
server_key_version: "encryptionkey@0"
I20260806 01:17:50.970999 18407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 01:17:50.974959 18435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:50.976630 18407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:17:50.976848 18407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "e4ce8aa66b2b4b348b7bf5340446f02f"
format_stamp: "Formatted at 2026-08-06 01:17:50 on dist-test-slave-7nm7"
server_key: "bb813128446ec54169bd95e1a6fadc36"
server_key_iv: "2a24daa8a83d44e5a6974e9c509772f5"
server_key_version: "encryptionkey@0"
I20260806 01:17:50.977070 18407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:17:50.999213 18407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:17:51.004978 18407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:17:51.005354 18407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:17:51.006528 18407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:17:51.008929 18407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:17:51.009043 18407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:51.009258 18407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:17:51.009334 18407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:17:51.052172 18407 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:43863
I20260806 01:17:51.052218 18600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:43863 every 8 connection(s)
I20260806 01:17:51.054061 18407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:17:51.064479 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 18407
I20260806 01:17:51.064718 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:17:51.065230 26546 external_mini_cluster.cc:1428] Setting key 91ab1b026e44ef6b4397bfcb8cd0f61c
Aug 06 01:17:51 dist-test-slave-7nm7 krb5kdc[17842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979070, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:17:51.078796 18604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.057708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:37325 (local address 127.25.236.190:35573)
0806 01:17:51.058197 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:51.058203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:51.059261 (+ 1058us) server_negotiation.cc:407] Connection header received
0806 01:17:51.061303 (+ 2042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:51.061311 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:51.061456 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:51.061647 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:51.063815 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.064713 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.066609 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.067020 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.067162 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.071024 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:51.071029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:51.071033 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:51.071067 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:51.074855 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:51.076086 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:51.076094 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:51.076097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:51.076276 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:51.076839 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:51.076843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:51.076845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:51.077142 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:51.077324 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:51.078072 (+ 748us) server_negotiation.cc:299] Negotiation successful
0806 01:17:51.078367 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":227,"threads_started":1}
I20260806 01:17:51.080240 18605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.058332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:37325)
0806 01:17:51.059077 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:51.059125 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:51.060870 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:51.061996 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:51.062007 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:51.062928 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:17:51.063574 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:51.063597 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.064912 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.064916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:51.066448 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:51.066461 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.067270 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.067280 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:51.067523 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.068237 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:17:51.068267 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:17:51.070782 (+ 2515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:17:51.075121 (+ 4339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:51.075136 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:51.075170 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:51.075877 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:51.076417 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:17:51.076423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:17:51.076427 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:17:51.076693 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:17:51.077281 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:17:51.077291 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:17:51.077694 (+ 403us) client_negotiation.cc:769] Sending connection context
0806 01:17:51.077979 (+ 285us) client_negotiation.cc:240] Negotiation successful
0806 01:17:51.078602 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":285,"threads_started":1}
I20260806 01:17:51.081928 18601 heartbeater.cc:347] Connected to a master server at 127.25.236.190:35573
I20260806 01:17:51.082414 18601 heartbeater.cc:464] Registering TS with master...
I20260806 01:17:51.083277 18601 heartbeater.cc:511] Master 127.25.236.190:35573 requested a full tablet report, sending...
I20260806 01:17:51.084991 17909 ts_manager.cc:194] Registered new tserver with Master: e4ce8aa66b2b4b348b7bf5340446f02f (127.25.236.131:43863)
I20260806 01:17:51.086078 17909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:37325
I20260806 01:17:51.094069 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:17:51.113459 18604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.099583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46554 (local address 127.25.236.190:35573)
0806 01:17:51.100036 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:51.100045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:51.100432 (+ 387us) server_negotiation.cc:407] Connection header received
0806 01:17:51.100654 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:51.100658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:51.100742 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:51.100857 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:51.102421 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.103178 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.104618 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.105031 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.105181 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.107769 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:51.107774 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:51.107777 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:51.107807 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:51.110530 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:51.111309 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:51.111317 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:51.111321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:51.111464 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:51.111879 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:51.111885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:51.111888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:51.112245 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:51.112510 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:51.112866 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 01:17:51.113120 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260806 01:17:51.118460 17909 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:17:51.125676 17909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:17:51.159392 18618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.140595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55456 (local address 127.25.236.131:43863)
0806 01:17:51.148860 (+ 8265us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:51.148875 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:51.148924 (+ 49us) server_negotiation.cc:407] Connection header received
0806 01:17:51.149062 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:51.149073 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:51.150434 (+ 1361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:51.150683 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:51.152034 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.153712 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.156369 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.157923 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.158118 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.158460 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:51.158848 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 01:17:51.159014 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8002,"thread_start_us":7773,"threads_started":1}
I20260806 01:17:51.159392 18617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.140356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:43863 (local address 127.0.0.1:55456)
0806 01:17:51.141605 (+ 1249us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:51.141655 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:51.141848 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:51.151030 (+ 9182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:51.151042 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:51.151088 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:51.151811 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:51.151825 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.153894 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.153900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:51.156177 (+ 2277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:51.156191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.158377 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.158389 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:51.158591 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.158631 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 01:17:51.158789 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 01:17:51.158991 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"thread_start_us":1043,"threads_started":1}
I20260806 01:17:51.163612 18504 tablet_service.cc:1467] Processing CreateTablet for tablet 4e26c7b36ecf4911a5e6ba61f0b5f3cd (DEFAULT_TABLE table=test-table [id=7f52c28576284701ac24fbc177d71e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:17:51.164661 18619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.141354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:43759 (local address 127.0.0.1:44674)
0806 01:17:51.149541 (+ 8187us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:51.149567 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:51.149759 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:51.154592 (+ 4833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:51.154600 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:51.154632 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:51.156661 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:51.156672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.159461 (+ 2789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.159469 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:51.161753 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:51.161766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.163713 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.163719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:51.163899 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.163937 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 01:17:51.164030 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 01:17:51.164209 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7915,"mutex_wait_us":1502,"thread_start_us":7739,"threads_started":1}
I20260806 01:17:51.164669 18620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.141928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44674 (local address 127.25.236.130:43759)
0806 01:17:51.150484 (+ 8556us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:51.150496 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:51.150535 (+ 39us) server_negotiation.cc:407] Connection header received
0806 01:17:51.150664 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:51.150676 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:51.150952 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:51.151224 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:51.157031 (+ 5807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.159228 (+ 2197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.161935 (+ 2707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.163471 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.163646 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.163834 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:51.164093 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 01:17:51.164303 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8327,"thread_start_us":8152,"threads_started":1}
I20260806 01:17:51.165354 18504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e26c7b36ecf4911a5e6ba61f0b5f3cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:51.169179 18301 tablet_service.cc:1467] Processing CreateTablet for tablet 4e26c7b36ecf4911a5e6ba61f0b5f3cd (DEFAULT_TABLE table=test-table [id=7f52c28576284701ac24fbc177d71e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:17:51.170132 18621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.142168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:41553 (local address 127.0.0.1:53980)
0806 01:17:51.153018 (+ 10850us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:51.153054 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:51.153213 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:51.161443 (+ 8230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:51.161448 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:51.161476 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:51.162013 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:51.162025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.163990 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.163999 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:51.166832 (+ 2833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:51.166848 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.169042 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.169048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:51.169241 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.169273 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:17:51.169476 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 01:17:51.169625 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10650,"thread_start_us":11163,"threads_started":1}
I20260806 01:17:51.170655 18622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.142457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53980 (local address 127.25.236.129:41553)
0806 01:17:51.160764 (+ 18307us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:51.160774 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:51.160809 (+ 35us) server_negotiation.cc:407] Connection header received
0806 01:17:51.160930 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:51.160940 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:51.161202 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:51.162012 (+ 810us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:51.162273 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.163785 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.167020 (+ 3235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.168746 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.168963 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.169135 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:51.169556 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 01:17:51.169728 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17873,"thread_start_us":17985,"threads_started":1}
I20260806 01:17:51.170882 18301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e26c7b36ecf4911a5e6ba61f0b5f3cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:51.175218 18084 tablet_service.cc:1467] Processing CreateTablet for tablet 4e26c7b36ecf4911a5e6ba61f0b5f3cd (DEFAULT_TABLE table=test-table [id=7f52c28576284701ac24fbc177d71e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:17:51.177073 18084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e26c7b36ecf4911a5e6ba61f0b5f3cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:17:51.182704 18623 tablet_bootstrap.cc:492] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f: Bootstrap starting.
I20260806 01:17:51.187794 18623 tablet_bootstrap.cc:654] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:51.189908 18623 log.cc:824] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:51.191252 18624 tablet_bootstrap.cc:492] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349: Bootstrap starting.
I20260806 01:17:51.193389 18625 tablet_bootstrap.cc:492] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb: Bootstrap starting.
I20260806 01:17:51.196396 18624 tablet_bootstrap.cc:654] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:51.197705 18624 log.cc:824] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:51.198478 18623 tablet_bootstrap.cc:492] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f: No bootstrap required, opened a new log
I20260806 01:17:51.198802 18623 ts_tablet_manager.cc:1397] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.006s
I20260806 01:17:51.198884 18625 tablet_bootstrap.cc:654] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb: Neither blocks nor log segments found. Creating new log.
I20260806 01:17:51.200167 18625 log.cc:824] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb: Log is configured to *not* fsync() on all Append() calls
I20260806 01:17:51.200446 18624 tablet_bootstrap.cc:492] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349: No bootstrap required, opened a new log
I20260806 01:17:51.200729 18624 ts_tablet_manager.cc:1397] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260806 01:17:51.203316 18625 tablet_bootstrap.cc:492] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb: No bootstrap required, opened a new log
I20260806 01:17:51.203579 18625 ts_tablet_manager.cc:1397] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 01:17:51.209446 18624 raft_consensus.cc:348] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } }
I20260806 01:17:51.209568 18623 raft_consensus.cc:348] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } }
I20260806 01:17:51.209911 18624 raft_consensus.cc:374] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:51.210019 18623 raft_consensus.cc:374] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:51.210017 18624 raft_consensus.cc:729] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b98987123d3e4758abbf3b5279a7d349, State: Initialized, Role: FOLLOWER
I20260806 01:17:51.210129 18623 raft_consensus.cc:729] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4ce8aa66b2b4b348b7bf5340446f02f, State: Initialized, Role: FOLLOWER
I20260806 01:17:51.210747 18624 consensus_queue.cc:260] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } }
I20260806 01:17:51.210855 18623 consensus_queue.cc:260] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } }
I20260806 01:17:51.212005 18398 heartbeater.cc:503] Master 127.25.236.190:35573 was elected leader, sending a full tablet report...
I20260806 01:17:51.212666 18624 ts_tablet_manager.cc:1428] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260806 01:17:51.214494 18625 raft_consensus.cc:348] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } }
I20260806 01:17:51.214948 18625 raft_consensus.cc:374] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:17:51.214939 18601 heartbeater.cc:503] Master 127.25.236.190:35573 was elected leader, sending a full tablet report...
I20260806 01:17:51.215046 18625 raft_consensus.cc:729] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e711d05b3eda42a1821405b09d4fa7bb, State: Initialized, Role: FOLLOWER
I20260806 01:17:51.215680 18623 ts_tablet_manager.cc:1428] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f: Time spent starting tablet: real 0.017s user 0.011s sys 0.004s
I20260806 01:17:51.215744 18625 consensus_queue.cc:260] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } }
I20260806 01:17:51.218608 18625 ts_tablet_manager.cc:1428] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260806 01:17:51.223888 18640 raft_consensus.cc:482] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:17:51.224123 18640 raft_consensus.cc:504] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } }
I20260806 01:17:51.225908 18640 leader_election.cc:288] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4ce8aa66b2b4b348b7bf5340446f02f (127.25.236.131:43863), b98987123d3e4758abbf3b5279a7d349 (127.25.236.130:43759)
I20260806 01:17:51.234421 18641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.226372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:43863 (local address 127.25.236.129:33517)
0806 01:17:51.227283 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:51.227313 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:51.227485 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:51.227854 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:51.227859 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:51.227916 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:51.228487 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:51.228522 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.229902 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.229907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:51.231714 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:51.231727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.233322 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.233331 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:51.233524 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.233559 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 01:17:51.233678 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 01:17:51.233785 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"mutex_wait_us":37,"thread_start_us":464,"threads_started":1}
I20260806 01:17:51.234431 18618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.226421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:33517 (local address 127.25.236.131:43863)
0806 01:17:51.226650 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:51.226657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:51.227434 (+ 777us) server_negotiation.cc:407] Connection header received
0806 01:17:51.227634 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:51.227640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:51.227720 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:51.227823 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:51.228767 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.229732 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.231883 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.232919 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.233054 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.233111 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:51.233733 (+ 622us) server_negotiation.cc:299] Negotiation successful
0806 01:17:51.233891 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:17:51.234742 18620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.226412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:53673 (local address 127.25.236.130:43759)
0806 01:17:51.226658 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:51.226667 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:51.227824 (+ 1157us) server_negotiation.cc:407] Connection header received
0806 01:17:51.227978 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:51.227984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:51.228065 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:51.228183 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:17:51.229029 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.229766 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.232303 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.233376 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.233547 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.233639 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:51.234306 (+ 667us) server_negotiation.cc:299] Negotiation successful
0806 01:17:51.234469 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 01:17:51.235632 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.226828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:43759 (local address 127.25.236.129:53673)
0806 01:17:51.227690 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0806 01:17:51.227707 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:17:51.227844 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:17:51.228377 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:17:51.228381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:17:51.228410 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:17:51.228830 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:51.228847 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.229956 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.229960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:51.232030 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:17:51.232046 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.233670 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.233683 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:17:51.233869 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.233910 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 01:17:51.234918 (+ 1008us) client_negotiation.cc:240] Negotiation successful
0806 01:17:51.235069 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"mutex_wait_us":4,"thread_start_us":468,"threads_started":1}
I20260806 01:17:51.235697 18533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e26c7b36ecf4911a5e6ba61f0b5f3cd" candidate_uuid: "e711d05b3eda42a1821405b09d4fa7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" is_pre_election: true
I20260806 01:17:51.236626 18533 raft_consensus.cc:2454] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e711d05b3eda42a1821405b09d4fa7bb in term 0.
I20260806 01:17:51.237004 18331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e26c7b36ecf4911a5e6ba61f0b5f3cd" candidate_uuid: "e711d05b3eda42a1821405b09d4fa7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b98987123d3e4758abbf3b5279a7d349" is_pre_election: true
I20260806 01:17:51.237495 18018 leader_election.cc:302] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e4ce8aa66b2b4b348b7bf5340446f02f, e711d05b3eda42a1821405b09d4fa7bb; no voters:
I20260806 01:17:51.237703 18331 raft_consensus.cc:2454] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e711d05b3eda42a1821405b09d4fa7bb in term 0.
I20260806 01:17:51.237999 18640 raft_consensus.cc:2781] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:17:51.238363 18640 raft_consensus.cc:482] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:17:51.238526 18640 raft_consensus.cc:3037] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:51.240784 18640 raft_consensus.cc:504] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } }
I20260806 01:17:51.241413 18640 leader_election.cc:288] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [CANDIDATE]: Term 1 election: Requested vote from peers e4ce8aa66b2b4b348b7bf5340446f02f (127.25.236.131:43863), b98987123d3e4758abbf3b5279a7d349 (127.25.236.130:43759)
I20260806 01:17:51.241979 18331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e26c7b36ecf4911a5e6ba61f0b5f3cd" candidate_uuid: "e711d05b3eda42a1821405b09d4fa7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b98987123d3e4758abbf3b5279a7d349"
I20260806 01:17:51.241979 18533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e26c7b36ecf4911a5e6ba61f0b5f3cd" candidate_uuid: "e711d05b3eda42a1821405b09d4fa7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f"
I20260806 01:17:51.242200 18331 raft_consensus.cc:3037] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:51.242199 18533 raft_consensus.cc:3037] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:17:51.245537 18533 raft_consensus.cc:2454] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e711d05b3eda42a1821405b09d4fa7bb in term 1.
I20260806 01:17:51.245532 18331 raft_consensus.cc:2454] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e711d05b3eda42a1821405b09d4fa7bb in term 1.
I20260806 01:17:51.245992 18021 leader_election.cc:302] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b98987123d3e4758abbf3b5279a7d349, e711d05b3eda42a1821405b09d4fa7bb; no voters:
I20260806 01:17:51.246466 18640 raft_consensus.cc:2781] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:17:51.246932 18640 raft_consensus.cc:686] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [term 1 LEADER]: Becoming Leader. State: Replica: e711d05b3eda42a1821405b09d4fa7bb, State: Running, Role: LEADER
I20260806 01:17:51.247398 18640 consensus_queue.cc:237] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } }
I20260806 01:17:51.253819 17907 catalog_manager.cc:5495] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb reported cstate change: term changed from 0 to 1, leader changed from <none> to e711d05b3eda42a1821405b09d4fa7bb (127.25.236.129). New cstate: current_term: 1 leader_uuid: "e711d05b3eda42a1821405b09d4fa7bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } health_report { overall_health: UNKNOWN } } }
I20260806 01:17:51.294762 18622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.289064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53986 (local address 127.25.236.129:41553)
0806 01:17:51.289327 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:51.289335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:51.289355 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:17:51.289619 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:51.289623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:51.289707 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:51.289865 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:17:51.290274 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.290986 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.292484 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.292898 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.293034 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.293550 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:17:51.294090 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:17:51.294279 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:51.294404 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 01:17:51.294490 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 01:17:51.305191 18331 raft_consensus.cc:1261] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349 [term 1 FOLLOWER]: Refusing update from remote peer e711d05b3eda42a1821405b09d4fa7bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:17:51.305193 18533 raft_consensus.cc:1261] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f [term 1 FOLLOWER]: Refusing update from remote peer e711d05b3eda42a1821405b09d4fa7bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 01:17:51.306152 18602 tablet.cc:2369] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:17:51.306348 18647 consensus_queue.cc:1035] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:17:51.306914 18640 consensus_queue.cc:1035] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:17:51.320122 18652 mvcc.cc:205] Tried to move back new op lower bound from 7315370276053032960 to 7315370275832807424. Current Snapshot: MvccSnapshot[applied={T|T < 7315370276053032960}]
I20260806 01:17:51.342075 17908 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:17:51.342475 17908 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:17:51.346805 17908 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Sending DeleteTablet for 3 replicas of tablet 4e26c7b36ecf4911a5e6ba61f0b5f3cd
I20260806 01:17:51.348598 18504 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e26c7b36ecf4911a5e6ba61f0b5f3cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:17:51 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:55456
I20260806 01:17:51.348628 18301 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e26c7b36ecf4911a5e6ba61f0b5f3cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:17:51 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:44674
I20260806 01:17:51.348628 18084 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e26c7b36ecf4911a5e6ba61f0b5f3cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:17:51 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:53980
I20260806 01:17:51.352739 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 18000
I20260806 01:17:51.363389 18666 tablet_replica.cc:330] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f: stopping tablet replica
I20260806 01:17:51.363893 18668 tablet_replica.cc:330] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349: stopping tablet replica
I20260806 01:17:51.364260 18669 tablet_replica.cc:330] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb: stopping tablet replica
I20260806 01:17:51.364444 18668 raft_consensus.cc:2229] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:17:51.364845 18669 raft_consensus.cc:2229] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:17:51.365046 18668 raft_consensus.cc:2258] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:17:51.366279 18669 raft_consensus.cc:2258] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:17:51.366489 18666 raft_consensus.cc:2229] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:17:51.367033 18666 raft_consensus.cc:2258] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:17:51.367903 18668 ts_tablet_manager.cc:1905] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:17:51.370541 18669 ts_tablet_manager.cc:1905] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:17:51.371177 18666 ts_tablet_manager.cc:1905] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:17:51.376096 18668 ts_tablet_manager.cc:1918] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:17:51.376294 18668 log.cc:1196] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/4e26c7b36ecf4911a5e6ba61f0b5f3cd
I20260806 01:17:51.377225 18668 ts_tablet_manager.cc:1939] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P b98987123d3e4758abbf3b5279a7d349: Deleting consensus metadata
I20260806 01:17:51.378778 18666 ts_tablet_manager.cc:1918] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:17:51.379047 18666 log.cc:1196] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/4e26c7b36ecf4911a5e6ba61f0b5f3cd
I20260806 01:17:51.379562 17893 catalog_manager.cc:4841] TS b98987123d3e4758abbf3b5279a7d349 (127.25.236.130:43759): tablet 4e26c7b36ecf4911a5e6ba61f0b5f3cd (table test-table [id=7f52c28576284701ac24fbc177d71e9b]) successfully deleted
I20260806 01:17:51.380155 18666 ts_tablet_manager.cc:1939] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e4ce8aa66b2b4b348b7bf5340446f02f: Deleting consensus metadata
Aug 06 01:17:51 dist-test-slave-7nm7 krb5kdc[17842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:17:51.382296 17891 catalog_manager.cc:4841] TS e4ce8aa66b2b4b348b7bf5340446f02f (127.25.236.131:43863): tablet 4e26c7b36ecf4911a5e6ba61f0b5f3cd (table test-table [id=7f52c28576284701ac24fbc177d71e9b]) successfully deleted
I20260806 01:17:51.384663 18669 ts_tablet_manager.cc:1918] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:17:51.384925 18669 log.cc:1196] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/4e26c7b36ecf4911a5e6ba61f0b5f3cd
I20260806 01:17:51.385639 18669 ts_tablet_manager.cc:1939] T 4e26c7b36ecf4911a5e6ba61f0b5f3cd P e711d05b3eda42a1821405b09d4fa7bb: Deleting consensus metadata
I20260806 01:17:51.387784 17891 catalog_manager.cc:4841] TS e711d05b3eda42a1821405b09d4fa7bb (127.25.236.129:41553): tablet 4e26c7b36ecf4911a5e6ba61f0b5f3cd (table test-table [id=7f52c28576284701ac24fbc177d71e9b]) successfully deleted
I20260806 01:17:51.392361 18622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.354316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53998 (local address 127.25.236.129:41553)
0806 01:17:51.354562 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:51.354572 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:51.367797 (+ 13225us) server_negotiation.cc:407] Connection header received
0806 01:17:51.368045 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:51.368050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:51.368138 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:51.368241 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:51.370412 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.371583 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.374378 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.375399 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.375519 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.383724 (+ 8205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:51.383744 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:51.383764 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:51.383814 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:51.388071 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:51.388847 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:51.388856 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:51.388858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:51.389007 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:51.389900 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:51.389906 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:51.389909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:51.390560 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:51.391185 (+ 625us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:51.391687 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 01:17:51.391996 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:17:51.393113 18173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:51.499183 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 18000
I20260806 01:17:51.527359 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 18191
Aug 06 01:17:51 dist-test-slave-7nm7 krb5kdc[17842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:17:51.544752 18620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.528407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.25.236.130:43759)
0806 01:17:51.528581 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:51.528585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:51.528603 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:17:51.528663 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:51.528667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:51.528728 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:51.528842 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:51.530171 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.531535 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.532964 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.533644 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.533755 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.538591 (+ 4836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:51.538608 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:51.538626 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:51.538665 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:51.541657 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:51.542492 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:51.542500 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:51.542503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:51.542639 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:51.543048 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:51.543053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:51.543056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:51.543479 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:51.543798 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:51.544120 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 01:17:51.544364 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:17:51.545451 18393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:51.644555 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 18191
I20260806 01:17:51.670083 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 18407
Aug 06 01:17:51 dist-test-slave-7nm7 krb5kdc[17842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:17:51.687935 18618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:17:51.670959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55466 (local address 127.25.236.131:43863)
0806 01:17:51.671345 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0806 01:17:51.671356 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:17:51.671401 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:17:51.671500 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:17:51.671505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:17:51.671604 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:17:51.671736 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:17:51.673243 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.674431 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.676339 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:17:51.676765 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:17:51.676880 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:17:51.682031 (+ 5151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:17:51.682048 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:17:51.682065 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:17:51.682131 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:17:51.684953 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:51.685753 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:51.685761 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:51.685763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:51.685896 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:17:51.686327 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:17:51.686331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:17:51.686333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:17:51.686762 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:17:51.687001 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 01:17:51.687377 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 01:17:51.687548 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:17:51.688689 18595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:51.783664 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 18407
I20260806 01:17:51.813570 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 17864
I20260806 01:17:51.814900 17949 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:17:51.928680 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 17864
I20260806 01:18:11.960835 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:35573
--webserver_interface=127.25.236.190
--webserver_port=41691
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:35573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:18:12.349825 18682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:12.350121 18682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:12.350212 18682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:12.357553 18682 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 01:18:12.357654 18682 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 01:18:12.357693 18682 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 01:18:12.357738 18682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:18:12.357795 18682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:12.357860 18682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:12.357905 18682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:18:12.357954 18682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:18:12.368283 18682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:35573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:35573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=41691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:12.369519 18682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:12.370798 18682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:12.397934 18694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:12.397971 18696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:12.398025 18693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:12.399000 18682 server_base.cc:1034] running on GCE node
I20260806 01:18:12.399938 18682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:12.401782 18682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:12.403035 18682 hybrid_clock.cc:648] HybridClock initialized: now 1785979092402938 us; error 62 us; skew 500 ppm
I20260806 01:18:12.403704 18682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:18:12 dist-test-slave-7nm7 krb5kdc[17842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979092, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:18:12.412803 18682 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:18:12.415313 18682 webserver.cc:460] Webserver started at http://127.25.236.190:41691/ using document root <none> and password file <none>
I20260806 01:18:12.416230 18682 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:12.416385 18682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:12.425118 18682 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 01:18:12.429095 18706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:12.430773 18682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:18:12.431000 18682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "28b759deabf24687960b360cdebb0ecc"
format_stamp: "Formatted at 2026-08-06 01:17:49 on dist-test-slave-7nm7"
server_key: "523a7de0a792a90de83d98afecf17bf9"
server_key_iv: "970d187d5fe9e955cf31b19cdb889bf7"
server_key_version: "encryptionkey@0"
I20260806 01:18:12.432291 18682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:12.475534 18682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:12.480861 18682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:12.481225 18682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:12.504988 18682 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:35573
I20260806 01:18:12.505149 18777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:35573 every 8 connection(s)
I20260806 01:18:12.507400 18682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:18:12.510512 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 18682
I20260806 01:18:12.511448 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:41553
--local_ip_for_outbound_sockets=127.25.236.129
--tserver_master_addrs=127.25.236.190:35573
--webserver_port=43869
--webserver_interface=127.25.236.129
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:18:12.516207 18778 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:18:12.519840 18778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Bootstrap starting.
I20260806 01:18:12.544093 18778 log.cc:824] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Log is configured to *not* fsync() on all Append() calls
I20260806 01:18:12.559221 18778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:18:12.560634 18778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Bootstrap complete.
I20260806 01:18:12.575809 18778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } }
I20260806 01:18:12.576802 18778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b759deabf24687960b360cdebb0ecc, State: Initialized, Role: FOLLOWER
I20260806 01:18:12.577555 18778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } }
I20260806 01:18:12.577859 18778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:18:12.577996 18778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:18:12.578613 18778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:18:12.583951 18778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } }
I20260806 01:18:12.584699 18778 leader_election.cc:302] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28b759deabf24687960b360cdebb0ecc; no voters:
I20260806 01:18:12.585584 18778 leader_election.cc:288] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 01:18:12.585714 18782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:18:12.588163 18782 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 2 LEADER]: Becoming Leader. State: Replica: 28b759deabf24687960b360cdebb0ecc, State: Running, Role: LEADER
I20260806 01:18:12.588927 18782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } }
I20260806 01:18:12.589190 18778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:18:12.591619 18783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "28b759deabf24687960b360cdebb0ecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } } }
I20260806 01:18:12.591985 18783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: This master's current role is: LEADER
I20260806 01:18:12.592602 18784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28b759deabf24687960b360cdebb0ecc. Latest consensus state: current_term: 2 leader_uuid: "28b759deabf24687960b360cdebb0ecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } } }
I20260806 01:18:12.592967 18784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: This master's current role is: LEADER
I20260806 01:18:12.598544 18790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:18:12.607311 18790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:18:12.609090 18790 catalog_manager.cc:1259] Loaded cluster ID: eaffb1d7f802451b923b56bb73c3c423
I20260806 01:18:12.609406 18790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:18:12.614074 18790 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:18:12.616736 18790 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Loaded TSK: 0
I20260806 01:18:12.653951 18790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Generated new TSK 1
I20260806 01:18:12.654520 18790 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 01:18:12.934720 18780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:12.935098 18780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:12.935281 18780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:12.945168 18780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:12.945370 18780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:12.945578 18780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:18:12.957175 18780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:41553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=43869
--tserver_master_addrs=127.25.236.190:35573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:12.958707 18780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:12.961043 18780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:12.979532 18812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:12.979982 18811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:12.980381 18814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:12.980894 18780 server_base.cc:1034] running on GCE node
I20260806 01:18:12.981588 18780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:12.982777 18780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:12.984028 18780 hybrid_clock.cc:648] HybridClock initialized: now 1785979092983958 us; error 54 us; skew 500 ppm
I20260806 01:18:12.984560 18780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:18:12 dist-test-slave-7nm7 krb5kdc[17842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979092, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:18:12.988430 18780 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:18:12.990365 18780 webserver.cc:460] Webserver started at http://127.25.236.129:43869/ using document root <none> and password file <none>
I20260806 01:18:12.991140 18780 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:12.991319 18780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:12.999542 18780 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 01:18:13.002714 18824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:13.004467 18780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:18:13.004781 18780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "e711d05b3eda42a1821405b09d4fa7bb"
format_stamp: "Formatted at 2026-08-06 01:17:49 on dist-test-slave-7nm7"
server_key: "3b2089675dce3be8ff4e72b4442b14e9"
server_key_iv: "78e283c4a2307d6b7bbb654c823478a7"
server_key_version: "encryptionkey@0"
I20260806 01:18:13.005873 18780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:13.027820 18780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:13.032359 18780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:13.032663 18780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:13.033730 18780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:18:13.036002 18780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:18:13.036203 18780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:13.036360 18780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:18:13.036422 18780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:13.079113 18780 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:41553
I20260806 01:18:13.079171 18981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:41553 every 8 connection(s)
I20260806 01:18:13.080602 18780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:18:13.091491 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 18780
I20260806 01:18:13.092638 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:43759
--local_ip_for_outbound_sockets=127.25.236.130
--tserver_master_addrs=127.25.236.190:35573
--webserver_port=34109
--webserver_interface=127.25.236.130
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:18:13 dist-test-slave-7nm7 krb5kdc[17842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979092, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:18:13.116873 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:13.083694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:58167)
0806 01:18:13.084581 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:13.084636 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:13.086460 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:13.087756 (+ 1296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:13.087763 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:13.088333 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:18:13.088854 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:13.088870 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:13.090820 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:13.090827 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:13.093353 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:13.093373 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:13.094994 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:13.095006 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:13.095382 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:13.096205 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:18:13.096239 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:18:13.106295 (+ 10056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:18:13.111734 (+ 5439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:18:13.111742 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:18:13.111755 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:18:13.112263 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:18:13.112866 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:18:13.112869 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:18:13.112872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:18:13.113073 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:18:13.114492 (+ 1419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:18:13.114499 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:18:13.114955 (+ 456us) client_negotiation.cc:769] Sending connection context
0806 01:18:13.115250 (+ 295us) client_negotiation.cc:240] Negotiation successful
0806 01:18:13.115784 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":392,"threads_started":1}
I20260806 01:18:13.117231 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:13.083452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:58167 (local address 127.25.236.190:35573)
0806 01:18:13.084002 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:13.084013 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:13.084776 (+ 763us) server_negotiation.cc:407] Connection header received
0806 01:18:13.086949 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:13.086961 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:13.087301 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:13.087584 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:13.089089 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:13.090556 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:13.093525 (+ 2969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:13.094654 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:13.094838 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:13.106701 (+ 11863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:13.106711 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:13.106729 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:13.106827 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:13.111532 (+ 4705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:13.112440 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:13.112452 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:13.112462 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:13.112708 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:13.113396 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:13.113405 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:13.113411 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:13.114269 (+ 858us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:13.114511 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:13.115560 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0806 01:18:13.116007 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":245,"threads_started":1}
I20260806 01:18:13.120987 18982 heartbeater.cc:347] Connected to a master server at 127.25.236.190:35573
I20260806 01:18:13.121413 18982 heartbeater.cc:464] Registering TS with master...
I20260806 01:18:13.122437 18982 heartbeater.cc:511] Master 127.25.236.190:35573 requested a full tablet report, sending...
I20260806 01:18:13.125000 18731 ts_manager.cc:194] Registered new tserver with Master: e711d05b3eda42a1821405b09d4fa7bb (127.25.236.129:41553)
I20260806 01:18:13.126575 18731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:58167
W20260806 01:18:13.471549 18986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:13.471812 18986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:13.471959 18986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:13.479291 18986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:13.479400 18986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:13.479586 18986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:18:13.489697 18986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:43759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=34109
--tserver_master_addrs=127.25.236.190:35573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:13.491384 18986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:13.492777 18986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:13.507277 18995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:13.507387 18998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:13.510365 18986 server_base.cc:1034] running on GCE node
W20260806 01:18:13.510756 18996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:13.511590 18986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:13.512638 18986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:13.513908 18986 hybrid_clock.cc:648] HybridClock initialized: now 1785979093513853 us; error 43 us; skew 500 ppm
I20260806 01:18:13.514336 18986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:18:13 dist-test-slave-7nm7 krb5kdc[17842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979093, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:18:13.517650 18986 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:18:13.519650 18986 webserver.cc:460] Webserver started at http://127.25.236.130:34109/ using document root <none> and password file <none>
I20260806 01:18:13.520546 18986 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:13.520781 18986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:13.526803 18986 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 01:18:13.529891 19008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:13.531421 18986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:18:13.531594 18986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "b98987123d3e4758abbf3b5279a7d349"
format_stamp: "Formatted at 2026-08-06 01:17:50 on dist-test-slave-7nm7"
server_key: "614cb15279c875d08b76f639637d7f07"
server_key_iv: "81651d8e41f37bd5157a1d9974728562"
server_key_version: "encryptionkey@0"
I20260806 01:18:13.532557 18986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:13.567382 18986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:13.573037 18986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:13.573374 18986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:13.574642 18986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:18:13.576227 18986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:18:13.576323 18986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:13.576444 18986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:18:13.576535 18986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:13.619091 18986 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:43759
I20260806 01:18:13.619176 19184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:43759 every 8 connection(s)
I20260806 01:18:13.620836 18986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:18:13.623564 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 18986
I20260806 01:18:13.624325 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:43863
--local_ip_for_outbound_sockets=127.25.236.131
--tserver_master_addrs=127.25.236.190:35573
--webserver_port=38785
--webserver_interface=127.25.236.131
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:18:13 dist-test-slave-7nm7 krb5kdc[17842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979093, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:18:13.645023 19187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:13.623899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:40633 (local address 127.25.236.190:35573)
0806 01:18:13.624364 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:13.624369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:13.625233 (+ 864us) server_negotiation.cc:407] Connection header received
0806 01:18:13.627258 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:13.627263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:13.627586 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:13.627985 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:13.630141 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:13.630958 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:13.632536 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:13.632968 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:13.633110 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:13.638396 (+ 5286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:13.638401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:13.638405 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:13.638442 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:13.641130 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:13.642215 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:13.642220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:13.642222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:13.642430 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:13.642955 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:13.642961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:13.642963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:13.643396 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:13.643510 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:13.644532 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0806 01:18:13.644748 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":256,"threads_started":1}
I20260806 01:18:13.646656 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:13.624294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:40633)
0806 01:18:13.625065 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:13.625118 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:13.626981 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:13.628251 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:13.628260 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:13.628786 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:18:13.629914 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:13.629939 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:13.631157 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:13.631161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:13.632379 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:13.632390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:13.633209 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:13.633216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:13.633451 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:13.634303 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:18:13.634331 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:18:13.638062 (+ 3731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:18:13.641289 (+ 3227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:18:13.641301 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:18:13.641322 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:18:13.641988 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:18:13.642549 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:18:13.642557 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:18:13.642563 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:18:13.642816 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:18:13.643547 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:18:13.643557 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:18:13.644055 (+ 498us) client_negotiation.cc:769] Sending connection context
0806 01:18:13.644449 (+ 394us) client_negotiation.cc:240] Negotiation successful
0806 01:18:13.645073 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":333,"threads_started":1}
I20260806 01:18:13.648772 19185 heartbeater.cc:347] Connected to a master server at 127.25.236.190:35573
I20260806 01:18:13.649194 19185 heartbeater.cc:464] Registering TS with master...
I20260806 01:18:13.650045 19185 heartbeater.cc:511] Master 127.25.236.190:35573 requested a full tablet report, sending...
I20260806 01:18:13.652559 18731 ts_manager.cc:194] Registered new tserver with Master: b98987123d3e4758abbf3b5279a7d349 (127.25.236.130:43759)
I20260806 01:18:13.653923 18731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:40633
W20260806 01:18:14.042133 19190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:14.042526 19190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:14.042634 19190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:14.050273 19190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:14.050416 19190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:14.050534 19190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:18:14.060631 19190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:43863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=38785
--tserver_master_addrs=127.25.236.190:35573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:14.061969 19190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:14.063313 19190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:14.077155 19197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:14.077104 19201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:14.077085 19203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:14.079234 19190 server_base.cc:1034] running on GCE node
I20260806 01:18:14.079921 19190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:14.080986 19190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:14.082418 19190 hybrid_clock.cc:648] HybridClock initialized: now 1785979094082197 us; error 52 us; skew 500 ppm
I20260806 01:18:14.082775 19190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:18:14 dist-test-slave-7nm7 krb5kdc[17842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979094, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:18:14.086102 19190 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:18:14.087757 19190 webserver.cc:460] Webserver started at http://127.25.236.131:38785/ using document root <none> and password file <none>
I20260806 01:18:14.088474 19190 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:14.088624 19190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:14.094040 19190 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:18:14.097746 19212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:14.099562 19190 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:18:14.099746 19190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "e4ce8aa66b2b4b348b7bf5340446f02f"
format_stamp: "Formatted at 2026-08-06 01:17:50 on dist-test-slave-7nm7"
server_key: "bb813128446ec54169bd95e1a6fadc36"
server_key_iv: "2a24daa8a83d44e5a6974e9c509772f5"
server_key_version: "encryptionkey@0"
I20260806 01:18:14.100726 19190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:14.119056 19190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:14.123840 19190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:14.124159 19190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:14.125452 19190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:18:14.127087 19190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:18:14.127185 19190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:14.127300 19190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:18:14.127377 19190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:14.130940 18982 heartbeater.cc:503] Master 127.25.236.190:35573 was elected leader, sending a full tablet report...
I20260806 01:18:14.164547 19190 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:43863
I20260806 01:18:14.164608 19378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:43863 every 8 connection(s)
I20260806 01:18:14.166117 19190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:18:14.169486 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 19190
Aug 06 01:18:14 dist-test-slave-7nm7 krb5kdc[17842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979094, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:18:14.190512 19382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.168930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:47721 (local address 127.25.236.190:35573)
0806 01:18:14.169512 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:14.169520 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:14.170953 (+ 1433us) server_negotiation.cc:407] Connection header received
0806 01:18:14.172777 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:14.172783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:14.172890 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:14.173012 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:14.174682 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.175927 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.177943 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.178592 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.178731 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.182720 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:14.182728 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:14.182734 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:14.182780 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:14.186183 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:14.187332 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:14.187336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:14.187339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:14.187473 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:14.187974 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:14.187979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:14.187982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:14.188384 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:14.188510 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:14.189639 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0806 01:18:14.189870 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":281,"threads_started":1}
I20260806 01:18:14.190973 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.170399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48618 (local address 127.25.236.190:35573)
0806 01:18:14.170907 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:14.170915 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:14.174871 (+ 3956us) server_negotiation.cc:407] Connection header received
0806 01:18:14.175044 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:14.175048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:14.175134 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:14.175238 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:14.177173 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.178278 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.179840 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.180264 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.180396 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.184045 (+ 3649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:14.184050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:14.184053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:14.184194 (+ 141us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:14.187574 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:14.188436 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:14.188469 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:14.188474 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:14.188624 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:14.189090 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:14.189188 (+ 98us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:14.189193 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:14.189552 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:14.189817 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:14.190350 (+ 533us) server_negotiation.cc:299] Negotiation successful
0806 01:18:14.190582 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":218,"threads_started":1}
I20260806 01:18:14.192066 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.169477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:47721)
0806 01:18:14.170795 (+ 1318us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:14.170843 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:14.172521 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:14.173257 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:14.173265 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:14.173765 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:18:14.174445 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:14.174474 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.176150 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.176155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:14.177775 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:14.177792 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.178860 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.178865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:14.179070 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.179911 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:18:14.179935 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:18:14.182470 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:18:14.186409 (+ 3939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:18:14.186420 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:18:14.186441 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:18:14.187056 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:18:14.187593 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:18:14.187597 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:18:14.187602 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:18:14.187831 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:18:14.188546 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:18:14.188558 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:18:14.188982 (+ 424us) client_negotiation.cc:769] Sending connection context
0806 01:18:14.189493 (+ 511us) client_negotiation.cc:240] Negotiation successful
0806 01:18:14.190014 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":394,"threads_started":1}
I20260806 01:18:14.193485 19379 heartbeater.cc:347] Connected to a master server at 127.25.236.190:35573
I20260806 01:18:14.193838 19379 heartbeater.cc:464] Registering TS with master...
I20260806 01:18:14.194659 19379 heartbeater.cc:511] Master 127.25.236.190:35573 requested a full tablet report, sending...
I20260806 01:18:14.196205 18730 ts_manager.cc:194] Registered new tserver with Master: e4ce8aa66b2b4b348b7bf5340446f02f (127.25.236.131:43863)
I20260806 01:18:14.197232 18730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:47721
I20260806 01:18:14.206887 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:18:14.213762 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.207829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59902 (local address 127.25.236.190:35573)
0806 01:18:14.208035 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:14.208043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:14.208521 (+ 478us) server_negotiation.cc:407] Connection header received
0806 01:18:14.208699 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:14.208703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:14.208839 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:14.208962 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:18:14.209379 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.210091 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.211896 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.212315 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.212410 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.212873 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:18:14.213250 (+ 377us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 01:18:14.213539 (+ 289us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59902: token expired
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 01:18:14.213892 19385 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59902: token expired
W20260806 01:18:14.214576 18610 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.25.236.190:35573)
I20260806 01:18:14.227698 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.215126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59908 (local address 127.25.236.190:35573)
0806 01:18:14.215269 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:14.215273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:14.215351 (+ 78us) server_negotiation.cc:407] Connection header received
0806 01:18:14.215456 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:14.215459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:14.215512 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:14.215597 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:14.216929 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.217707 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.219128 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.219515 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.219617 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.222466 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:14.222470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:14.222474 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:14.222503 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:14.225050 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:14.225711 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:14.225722 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:14.225725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:14.225857 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:14.226212 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:14.226216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:14.226218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:14.226606 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:14.226797 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:14.227145 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 01:18:14.227460 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 01:18:14.231567 18730 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:18:14.240394 18730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:18:14.273805 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.252607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:43759 (local address 127.0.0.1:42644)
0806 01:18:14.263788 (+ 11181us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:14.263847 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:14.264092 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:14.264979 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:14.264987 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:14.265025 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:18:14.265674 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:14.265719 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.267753 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.267761 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:14.270415 (+ 2654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:14.270431 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.272677 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.272684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:14.272875 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.272932 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 01:18:14.273089 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 01:18:14.273306 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10915,"mutex_wait_us":53,"thread_start_us":10732,"threads_started":1}
I20260806 01:18:14.274426 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.252607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:41553 (local address 127.0.0.1:60274)
0806 01:18:14.265540 (+ 12933us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:14.265572 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:14.265825 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:14.266773 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:14.266780 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:14.266808 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:18:14.267196 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:14.267208 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.269176 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.269181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:14.271354 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:14.271369 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.273503 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.273509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:14.273690 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.273732 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 01:18:14.273840 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 01:18:14.273985 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12608,"mutex_wait_us":143,"thread_start_us":12151,"threads_started":1}
I20260806 01:18:14.275288 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.252864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42644 (local address 127.25.236.130:43759)
0806 01:18:14.263165 (+ 10301us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:14.263177 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:14.264174 (+ 997us) server_negotiation.cc:407] Connection header received
0806 01:18:14.264428 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:14.264437 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:14.264701 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:14.264963 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:18:14.265912 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.267563 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.270614 (+ 3051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.272145 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.272313 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.272495 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:14.274332 (+ 1837us) server_negotiation.cc:299] Negotiation successful
0806 01:18:14.274536 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10111,"thread_start_us":9946,"threads_started":1}
I20260806 01:18:14.276263 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.253006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60274 (local address 127.25.236.129:41553)
0806 01:18:14.264715 (+ 11709us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:14.264726 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:14.265754 (+ 1028us) server_negotiation.cc:407] Connection header received
0806 01:18:14.266047 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:14.266055 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:14.266309 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:14.266513 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:18:14.267384 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.269002 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.271544 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.272978 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.273170 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.273323 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:14.275586 (+ 2263us) server_negotiation.cc:299] Negotiation successful
0806 01:18:14.275888 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11527,"thread_start_us":11377,"threads_started":1}
I20260806 01:18:14.280783 19086 tablet_service.cc:1467] Processing CreateTablet for tablet 07a74a3209664314b13b2e1a012713c2 (DEFAULT_TABLE table=test-table [id=1434917514204eef97a7b2ac516e4b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:18:14.282927 19086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a74a3209664314b13b2e1a012713c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:18:14.282812 18897 tablet_service.cc:1467] Processing CreateTablet for tablet 07a74a3209664314b13b2e1a012713c2 (DEFAULT_TABLE table=test-table [id=1434917514204eef97a7b2ac516e4b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:18:14.284471 18897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a74a3209664314b13b2e1a012713c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:18:14.285743 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.266296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:43863 (local address 127.0.0.1:39000)
0806 01:18:14.275022 (+ 8726us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:14.275061 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:14.275279 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:14.277988 (+ 2709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:14.277994 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:14.278030 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:18:14.278376 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:14.278391 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.279909 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.279914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:14.282075 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:14.282092 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.284929 (+ 2837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.284935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:14.285120 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.285155 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 01:18:14.285261 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 01:18:14.285422 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8361,"thread_start_us":11981,"threads_started":1}
I20260806 01:18:14.285864 19400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.266361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39000 (local address 127.25.236.131:43863)
0806 01:18:14.277421 (+ 11060us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:14.277427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:14.277461 (+ 34us) server_negotiation.cc:407] Connection header received
0806 01:18:14.277552 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:14.277558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:14.277763 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:14.277989 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:18:14.278566 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.279759 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.282362 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.284709 (+ 2347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.284951 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.285127 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:14.285320 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 01:18:14.285509 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10849,"thread_start_us":10662,"threads_started":1}
I20260806 01:18:14.289888 19284 tablet_service.cc:1467] Processing CreateTablet for tablet 07a74a3209664314b13b2e1a012713c2 (DEFAULT_TABLE table=test-table [id=1434917514204eef97a7b2ac516e4b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:18:14.291396 19284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a74a3209664314b13b2e1a012713c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:18:14.298314 19401 tablet_bootstrap.cc:492] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349: Bootstrap starting.
I20260806 01:18:14.303617 19403 tablet_bootstrap.cc:492] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb: Bootstrap starting.
I20260806 01:18:14.305583 19401 tablet_bootstrap.cc:654] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349: Neither blocks nor log segments found. Creating new log.
I20260806 01:18:14.307786 19404 tablet_bootstrap.cc:492] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f: Bootstrap starting.
I20260806 01:18:14.307750 19401 log.cc:824] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349: Log is configured to *not* fsync() on all Append() calls
I20260806 01:18:14.308748 19403 tablet_bootstrap.cc:654] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb: Neither blocks nor log segments found. Creating new log.
I20260806 01:18:14.310366 19403 log.cc:824] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb: Log is configured to *not* fsync() on all Append() calls
I20260806 01:18:14.311072 19404 tablet_bootstrap.cc:654] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f: Neither blocks nor log segments found. Creating new log.
I20260806 01:18:14.312240 19404 log.cc:824] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:18:14.315248 19404 tablet_bootstrap.cc:492] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f: No bootstrap required, opened a new log
I20260806 01:18:14.315559 19404 ts_tablet_manager.cc:1397] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 01:18:14.316653 19401 tablet_bootstrap.cc:492] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349: No bootstrap required, opened a new log
I20260806 01:18:14.316929 19401 ts_tablet_manager.cc:1397] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.006s
I20260806 01:18:14.317210 19403 tablet_bootstrap.cc:492] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb: No bootstrap required, opened a new log
I20260806 01:18:14.317481 19403 ts_tablet_manager.cc:1397] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260806 01:18:14.326001 19401 raft_consensus.cc:348] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } }
I20260806 01:18:14.326874 19401 raft_consensus.cc:374] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:18:14.327056 19401 raft_consensus.cc:729] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b98987123d3e4758abbf3b5279a7d349, State: Initialized, Role: FOLLOWER
I20260806 01:18:14.326813 19404 raft_consensus.cc:348] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } }
I20260806 01:18:14.327342 19404 raft_consensus.cc:374] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:18:14.327445 19404 raft_consensus.cc:729] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4ce8aa66b2b4b348b7bf5340446f02f, State: Initialized, Role: FOLLOWER
I20260806 01:18:14.327875 19401 consensus_queue.cc:260] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } }
I20260806 01:18:14.328020 19404 consensus_queue.cc:260] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } }
I20260806 01:18:14.328214 19403 raft_consensus.cc:348] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } }
I20260806 01:18:14.328697 19403 raft_consensus.cc:374] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:18:14.328824 19403 raft_consensus.cc:729] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e711d05b3eda42a1821405b09d4fa7bb, State: Initialized, Role: FOLLOWER
I20260806 01:18:14.329319 19185 heartbeater.cc:503] Master 127.25.236.190:35573 was elected leader, sending a full tablet report...
I20260806 01:18:14.329650 19403 consensus_queue.cc:260] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } }
I20260806 01:18:14.329983 19401 ts_tablet_manager.cc:1428] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260806 01:18:14.332913 19379 heartbeater.cc:503] Master 127.25.236.190:35573 was elected leader, sending a full tablet report...
I20260806 01:18:14.334848 19403 ts_tablet_manager.cc:1428] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb: Time spent starting tablet: real 0.017s user 0.013s sys 0.005s
I20260806 01:18:14.335278 19404 ts_tablet_manager.cc:1428] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f: Time spent starting tablet: real 0.020s user 0.014s sys 0.004s
W20260806 01:18:14.373044 19186 tablet.cc:2369] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:18:14.417882 19381 tablet.cc:2369] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:18:14.569418 19411 raft_consensus.cc:482] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:18:14.569775 19411 raft_consensus.cc:504] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } }
I20260806 01:18:14.571565 19411 leader_election.cc:288] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e711d05b3eda42a1821405b09d4fa7bb (127.25.236.129:41553), e4ce8aa66b2b4b348b7bf5340446f02f (127.25.236.131:43863)
I20260806 01:18:14.579387 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.572112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:58379 (local address 127.25.236.129:41553)
0806 01:18:14.572398 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:14.572405 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:14.572790 (+ 385us) server_negotiation.cc:407] Connection header received
0806 01:18:14.573045 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:14.573051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:14.573128 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:14.573199 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:18:14.574106 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.575118 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.577235 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.578444 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.578604 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.578689 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:14.579019 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 01:18:14.579116 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 01:18:14.579595 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.571993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:41553 (local address 127.25.236.130:58379)
0806 01:18:14.572609 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:14.572627 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:14.572845 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:14.573386 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:14.573390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:14.573415 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:18:14.573917 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:14.573931 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.575265 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.575270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:14.577069 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:14.577081 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.578666 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.578671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:14.578818 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.578861 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 01:18:14.578973 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 01:18:14.579130 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":57,"spinlock_wait_cycles":28416,"thread_start_us":319,"threads_started":1}
I20260806 01:18:14.579649 19400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.572950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:54021 (local address 127.25.236.131:43863)
0806 01:18:14.573145 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:14.573152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:14.573553 (+ 401us) server_negotiation.cc:407] Connection header received
0806 01:18:14.573732 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:14.573738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:14.573820 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:14.573930 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:18:14.574706 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.575571 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.577474 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.578738 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.578861 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.578909 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:14.579282 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 01:18:14.579414 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 01:18:14.579705 19417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.572719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:43863 (local address 127.25.236.130:54021)
0806 01:18:14.573427 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:14.573446 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:14.573595 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:14.573973 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:14.573977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:14.574035 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:18:14.574475 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:14.574490 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.575751 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.575756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:14.577305 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:14.577321 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.578936 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.578944 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:14.579124 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.579154 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 01:18:14.579242 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 01:18:14.579375 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":311,"threads_started":1}
I20260806 01:18:14.580740 19308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a74a3209664314b13b2e1a012713c2" candidate_uuid: "b98987123d3e4758abbf3b5279a7d349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" is_pre_election: true
I20260806 01:18:14.580922 18923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a74a3209664314b13b2e1a012713c2" candidate_uuid: "b98987123d3e4758abbf3b5279a7d349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e711d05b3eda42a1821405b09d4fa7bb" is_pre_election: true
I20260806 01:18:14.581358 19308 raft_consensus.cc:2454] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b98987123d3e4758abbf3b5279a7d349 in term 0.
I20260806 01:18:14.581452 18923 raft_consensus.cc:2454] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b98987123d3e4758abbf3b5279a7d349 in term 0.
I20260806 01:18:14.582054 19009 leader_election.cc:302] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b98987123d3e4758abbf3b5279a7d349, e4ce8aa66b2b4b348b7bf5340446f02f; no voters:
I20260806 01:18:14.582468 19411 raft_consensus.cc:2781] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:18:14.582613 19411 raft_consensus.cc:482] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:18:14.582736 19411 raft_consensus.cc:3037] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [term 0 FOLLOWER]: Advancing to term 1
W20260806 01:18:14.584381 18983 tablet.cc:2369] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:18:14.585017 19411 raft_consensus.cc:504] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } }
I20260806 01:18:14.585685 19411 leader_election.cc:288] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [CANDIDATE]: Term 1 election: Requested vote from peers e711d05b3eda42a1821405b09d4fa7bb (127.25.236.129:41553), e4ce8aa66b2b4b348b7bf5340446f02f (127.25.236.131:43863)
I20260806 01:18:14.586114 18923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a74a3209664314b13b2e1a012713c2" candidate_uuid: "b98987123d3e4758abbf3b5279a7d349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e711d05b3eda42a1821405b09d4fa7bb"
I20260806 01:18:14.586285 19308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a74a3209664314b13b2e1a012713c2" candidate_uuid: "b98987123d3e4758abbf3b5279a7d349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f"
I20260806 01:18:14.586346 18923 raft_consensus.cc:3037] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:18:14.586510 19308 raft_consensus.cc:3037] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:18:14.588737 19308 raft_consensus.cc:2454] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b98987123d3e4758abbf3b5279a7d349 in term 1.
I20260806 01:18:14.588774 18923 raft_consensus.cc:2454] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b98987123d3e4758abbf3b5279a7d349 in term 1.
I20260806 01:18:14.589347 19009 leader_election.cc:302] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b98987123d3e4758abbf3b5279a7d349, e4ce8aa66b2b4b348b7bf5340446f02f; no voters:
I20260806 01:18:14.589710 19411 raft_consensus.cc:2781] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:18:14.590320 19411 raft_consensus.cc:686] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [term 1 LEADER]: Becoming Leader. State: Replica: b98987123d3e4758abbf3b5279a7d349, State: Running, Role: LEADER
I20260806 01:18:14.590886 19411 consensus_queue.cc:237] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } } peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } }
I20260806 01:18:14.597191 18730 catalog_manager.cc:5495] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 reported cstate change: term changed from 0 to 1, leader changed from <none> to b98987123d3e4758abbf3b5279a7d349 (127.25.236.130). New cstate: current_term: 1 leader_uuid: "b98987123d3e4758abbf3b5279a7d349" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98987123d3e4758abbf3b5279a7d349" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 } health_report { overall_health: UNKNOWN } } }
I20260806 01:18:14.664261 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.658056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42654 (local address 127.25.236.130:43759)
0806 01:18:14.658432 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:14.658437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:14.658511 (+ 74us) server_negotiation.cc:407] Connection header received
0806 01:18:14.658741 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:14.658745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:14.658861 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:14.659208 (+ 347us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:18:14.659554 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.660303 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.661815 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.662268 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.662452 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.662975 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:18:14.663597 (+ 622us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:18:14.663748 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:14.663848 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 01:18:14.663919 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240}
I20260806 01:18:14.674053 18923 raft_consensus.cc:1261] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb [term 1 FOLLOWER]: Refusing update from remote peer b98987123d3e4758abbf3b5279a7d349: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:18:14.674724 19308 raft_consensus.cc:1261] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f [term 1 FOLLOWER]: Refusing update from remote peer b98987123d3e4758abbf3b5279a7d349: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:18:14.675163 19419 consensus_queue.cc:1035] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e711d05b3eda42a1821405b09d4fa7bb" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 41553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:18:14.675645 19411 consensus_queue.cc:1035] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4ce8aa66b2b4b348b7bf5340446f02f" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 43863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:18:14.687886 19424 mvcc.cc:205] Tried to move back new op lower bound from 7315370371771547648 to 7315370371448537088. Current Snapshot: MvccSnapshot[applied={T|T < 7315370371771547648}]
I20260806 01:18:14.705710 18728 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:18:14.706034 18728 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:18:14.709830 18728 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Sending DeleteTablet for 3 replicas of tablet 07a74a3209664314b13b2e1a012713c2
I20260806 01:18:14.711201 18897 tablet_service.cc:1514] Processing DeleteTablet for tablet 07a74a3209664314b13b2e1a012713c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:18:14 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:60274
I20260806 01:18:14.711427 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 18780
I20260806 01:18:14.711773 19086 tablet_service.cc:1514] Processing DeleteTablet for tablet 07a74a3209664314b13b2e1a012713c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:18:14 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:42644
I20260806 01:18:14.711977 19284 tablet_service.cc:1514] Processing DeleteTablet for tablet 07a74a3209664314b13b2e1a012713c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:18:14 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:39000
I20260806 01:18:14.721674 19445 tablet_replica.cc:330] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f: stopping tablet replica
I20260806 01:18:14.722285 19445 raft_consensus.cc:2229] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:18:14.722937 19445 raft_consensus.cc:2258] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:18:14.727509 19440 tablet_replica.cc:330] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb: stopping tablet replica
I20260806 01:18:14.728108 19440 raft_consensus.cc:2229] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:18:14.728701 19440 raft_consensus.cc:2258] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:18:14.729785 19441 tablet_replica.cc:330] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349: stopping tablet replica
I20260806 01:18:14.730192 19441 raft_consensus.cc:2229] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:18:14.730932 19441 raft_consensus.cc:2258] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:18:14.730962 19445 ts_tablet_manager.cc:1905] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:18:14.732901 19440 ts_tablet_manager.cc:1905] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:18:14.737980 19441 ts_tablet_manager.cc:1905] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:18:14.738965 19445 ts_tablet_manager.cc:1918] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:18:14.739148 19445 log.cc:1196] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/07a74a3209664314b13b2e1a012713c2
I20260806 01:18:14.739687 19445 ts_tablet_manager.cc:1939] T 07a74a3209664314b13b2e1a012713c2 P e4ce8aa66b2b4b348b7bf5340446f02f: Deleting consensus metadata
I20260806 01:18:14.741716 18707 catalog_manager.cc:4841] TS e4ce8aa66b2b4b348b7bf5340446f02f (127.25.236.131:43863): tablet 07a74a3209664314b13b2e1a012713c2 (table test-table [id=1434917514204eef97a7b2ac516e4b91]) successfully deleted
I20260806 01:18:14.744454 19441 ts_tablet_manager.cc:1918] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:18:14.744675 19441 log.cc:1196] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/07a74a3209664314b13b2e1a012713c2
I20260806 01:18:14.745314 19441 ts_tablet_manager.cc:1939] T 07a74a3209664314b13b2e1a012713c2 P b98987123d3e4758abbf3b5279a7d349: Deleting consensus metadata
I20260806 01:18:14.747409 18710 catalog_manager.cc:4841] TS b98987123d3e4758abbf3b5279a7d349 (127.25.236.130:43759): tablet 07a74a3209664314b13b2e1a012713c2 (table test-table [id=1434917514204eef97a7b2ac516e4b91]) successfully deleted
I20260806 01:18:14.750010 19440 ts_tablet_manager.cc:1918] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:18:14.750175 19440 log.cc:1196] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/07a74a3209664314b13b2e1a012713c2
I20260806 01:18:14.750795 19440 ts_tablet_manager.cc:1939] T 07a74a3209664314b13b2e1a012713c2 P e711d05b3eda42a1821405b09d4fa7bb: Deleting consensus metadata
I20260806 01:18:14.751798 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.712649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60278 (local address 127.25.236.129:41553)
0806 01:18:14.712831 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:14.712837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:14.723015 (+ 10178us) server_negotiation.cc:407] Connection header received
0806 01:18:14.723576 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:14.723582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:14.723672 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:14.723764 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:14.726932 (+ 3168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.728648 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.731156 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.731845 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.731948 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.735922 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:14.735939 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:14.735959 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:14.736003 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:14.747967 (+ 11964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:14.749122 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:14.749133 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:14.749136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:14.749313 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:14.749826 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:14.749832 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:14.749835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:14.750433 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:14.750780 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:14.751148 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 01:18:14.751407 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:18:14.752697 18976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:18:14.752789 18707 catalog_manager.cc:4841] TS e711d05b3eda42a1821405b09d4fa7bb (127.25.236.129:41553): tablet 07a74a3209664314b13b2e1a012713c2 (table test-table [id=1434917514204eef97a7b2ac516e4b91]) successfully deleted
I20260806 01:18:14.877070 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 18780
I20260806 01:18:14.909157 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 18986
I20260806 01:18:14.924746 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:14.910200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42664 (local address 127.25.236.130:43759)
0806 01:18:14.910424 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:14.910430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:14.910452 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:18:14.910506 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:14.910509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:14.910571 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:14.910704 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:14.912160 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.912956 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.914205 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:14.914843 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:14.914981 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:14.917853 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:14.917872 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:14.917889 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:14.917932 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:14.921510 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:14.922447 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:14.922454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:14.922457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:14.922590 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:14.922998 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:14.923003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:14.923006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:14.923401 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:14.923758 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:14.924108 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 01:18:14.924338 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 01:18:14.925550 19178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:18:15.046588 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 18986
I20260806 01:18:15.075117 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 19190
I20260806 01:18:15.089669 19400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:15.075866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39016 (local address 127.25.236.131:43863)
0806 01:18:15.076052 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:15.076059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:15.076105 (+ 46us) server_negotiation.cc:407] Connection header received
0806 01:18:15.076332 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:15.076336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:15.076401 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:15.076512 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:15.077955 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:15.078738 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:15.080146 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:15.080533 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:15.080670 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:15.083845 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:15.083858 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:15.083874 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:15.083915 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:15.086793 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:15.087493 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:15.087501 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:15.087503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:15.087638 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:15.088026 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:15.088031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:15.088034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:15.088411 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:15.088728 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:15.089051 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 01:18:15.089287 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:18:15.090565 19372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:18:15.183938 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 19190
I20260806 01:18:15.209010 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 18682
I20260806 01:18:15.210475 18773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:18:15.330169 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 18682
I20260806 01:18:35.364646 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:35573
--webserver_interface=127.25.236.190
--webserver_port=41691
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:35573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:18:35.737857 19450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:35.738147 19450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:35.738348 19450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:35.745790 19450 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 01:18:35.745901 19450 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 01:18:35.745972 19450 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 01:18:35.746050 19450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:18:35.746126 19450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:35.746220 19450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:35.746308 19450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:18:35.746410 19450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:18:35.756505 19450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:35573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:35573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=41691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:35.757879 19450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:35.759256 19450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:35.771482 19466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:35.771445 19461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:35.771519 19462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:35.772384 19450 server_base.cc:1034] running on GCE node
I20260806 01:18:35.773118 19450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:35.774505 19450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:35.775766 19450 hybrid_clock.cc:648] HybridClock initialized: now 1785979115775683 us; error 31 us; skew 500 ppm
I20260806 01:18:35.776173 19450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:18:35 dist-test-slave-7nm7 krb5kdc[17842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979115, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:18:35.779701 19450 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:18:35.781755 19450 webserver.cc:460] Webserver started at http://127.25.236.190:41691/ using document root <none> and password file <none>
I20260806 01:18:35.782634 19450 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:35.782732 19450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:35.788278 19450 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 01:18:35.791421 19477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:35.793030 19450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:18:35.793244 19450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "28b759deabf24687960b360cdebb0ecc"
format_stamp: "Formatted at 2026-08-06 01:17:49 on dist-test-slave-7nm7"
server_key: "523a7de0a792a90de83d98afecf17bf9"
server_key_iv: "970d187d5fe9e955cf31b19cdb889bf7"
server_key_version: "encryptionkey@0"
I20260806 01:18:35.794234 19450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:35.862223 19450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:35.867266 19450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:35.867612 19450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:35.886818 19450 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:35573
I20260806 01:18:35.886868 19547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:35573 every 8 connection(s)
I20260806 01:18:35.888377 19450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:18:35.891096 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 19450
I20260806 01:18:35.892171 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:41553
--local_ip_for_outbound_sockets=127.25.236.129
--tserver_master_addrs=127.25.236.190:35573
--webserver_port=43869
--webserver_interface=127.25.236.129
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:18:35.898561 19549 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:18:35.903919 19549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Bootstrap starting.
I20260806 01:18:35.926108 19549 log.cc:824] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Log is configured to *not* fsync() on all Append() calls
I20260806 01:18:35.943259 19549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:18:35.943909 19549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Bootstrap complete.
I20260806 01:18:35.954605 19549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } }
I20260806 01:18:35.955507 19549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b759deabf24687960b360cdebb0ecc, State: Initialized, Role: FOLLOWER
I20260806 01:18:35.956135 19549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } }
I20260806 01:18:35.956383 19549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:18:35.956504 19549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:18:35.956696 19549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 2 FOLLOWER]: Advancing to term 3
I20260806 01:18:35.960732 19549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } }
I20260806 01:18:35.961264 19549 leader_election.cc:302] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28b759deabf24687960b360cdebb0ecc; no voters:
I20260806 01:18:35.961997 19549 leader_election.cc:288] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 01:18:35.962149 19556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 3 FOLLOWER]: Leader election won for term 3
I20260806 01:18:35.963552 19556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [term 3 LEADER]: Becoming Leader. State: Replica: 28b759deabf24687960b360cdebb0ecc, State: Running, Role: LEADER
I20260806 01:18:35.964848 19556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } }
I20260806 01:18:35.966001 19549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:18:35.968077 19558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28b759deabf24687960b360cdebb0ecc. Latest consensus state: current_term: 3 leader_uuid: "28b759deabf24687960b360cdebb0ecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } } }
I20260806 01:18:35.968602 19558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: This master's current role is: LEADER
I20260806 01:18:35.968215 19557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "28b759deabf24687960b360cdebb0ecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b759deabf24687960b360cdebb0ecc" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 35573 } } }
I20260806 01:18:35.969277 19557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc [sys.catalog]: This master's current role is: LEADER
I20260806 01:18:35.979285 19562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:18:35.993660 19562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:18:35.995916 19562 catalog_manager.cc:1259] Loaded cluster ID: eaffb1d7f802451b923b56bb73c3c423
I20260806 01:18:35.996125 19562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:18:36.002413 19562 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:18:36.004873 19562 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Loaded TSK: 0
I20260806 01:18:36.005357 19562 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 28b759deabf24687960b360cdebb0ecc: Loaded TSK: 1
F20260806 01:18:36.006863 19562 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8bcd4379ad google::LogMessage::Fail() at ??:0
@ 0x7f8bcd43a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f8bcd4374de google::LogMessage::Flush() at ??:0
@ 0x7f8bcd438a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8bd119b7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f8bd118a73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f8be2e661c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f8be2e65783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f8be2ef12a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f8be2ef10c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f8be2f3b5e9 std::function<>::operator()() at ??:0
@ 0x7f8be2e69cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f8be2e67c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8be2eef610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f8be2eef439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8bd0083345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8bd008691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f8bd0086799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8bd0061eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8bd333b6db start_thread at ??:0
@ 0x7f8bcc2d161f clone at ??:0
W20260806 01:18:36.339387 19551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:36.339689 19551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:36.339787 19551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:36.348261 19551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:36.348549 19551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:36.348762 19551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:18:36.362003 19551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:41553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=43869
--tserver_master_addrs=127.25.236.190:35573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:36.363454 19551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:36.364840 19551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:36.378126 19625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:36.378371 19626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:36.378665 19628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:36.379419 19551 server_base.cc:1034] running on GCE node
I20260806 01:18:36.380071 19551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:36.381088 19551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:36.382372 19551 hybrid_clock.cc:648] HybridClock initialized: now 1785979116382303 us; error 69 us; skew 500 ppm
I20260806 01:18:36.382732 19551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:18:36 dist-test-slave-7nm7 krb5kdc[17842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979116, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:18:36.386909 19551 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:18:36.388621 19551 webserver.cc:460] Webserver started at http://127.25.236.129:43869/ using document root <none> and password file <none>
I20260806 01:18:36.389323 19551 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:36.389432 19551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:36.395258 19551 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 01:18:36.398489 19637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:36.400118 19551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:18:36.400419 19551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "e711d05b3eda42a1821405b09d4fa7bb"
format_stamp: "Formatted at 2026-08-06 01:17:49 on dist-test-slave-7nm7"
server_key: "3b2089675dce3be8ff4e72b4442b14e9"
server_key_iv: "78e283c4a2307d6b7bbb654c823478a7"
server_key_version: "encryptionkey@0"
I20260806 01:18:36.401429 19551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:36.415418 19551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:36.419891 19551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:36.420210 19551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:36.421484 19551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:18:36.422895 19551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:18:36.422984 19551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:36.423146 19551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:18:36.423249 19551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:36.459833 19551 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:41553
I20260806 01:18:36.459923 19794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:41553 every 8 connection(s)
I20260806 01:18:36.461297 19551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:18:36.466190 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:36.464227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:51497)
0806 01:18:36.465039 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:36.465316 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":331,"threads_started":1}
W20260806 01:18:36.467031 19795 heartbeater.cc:644] Failed to heartbeat to 127.25.236.190:35573 (0 consecutive failures): Network error: Failed to ping master at 127.25.236.190:35573: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
I20260806 01:18:36.467998 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:36.467630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:33383)
0806 01:18:36.467796 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:36.467880 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 01:18:36.469094 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:36.468597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:50321)
0806 01:18:36.468809 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:36.468926 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 01:18:36.469427 19795 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 01:18:36.469452 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 19551
I20260806 01:18:36.470119 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:43759
--local_ip_for_outbound_sockets=127.25.236.130
--tserver_master_addrs=127.25.236.190:35573
--webserver_port=34109
--webserver_interface=127.25.236.130
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:18:36.837765 19798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:36.838011 19798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:36.838104 19798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:36.845134 19798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:36.845229 19798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:36.845335 19798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:18:36.854591 19798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:43759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=34109
--tserver_master_addrs=127.25.236.190:35573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:36.855726 19798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:36.856911 19798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:36.869843 19806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:36.869969 19805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:36.870062 19810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:36.871241 19798 server_base.cc:1034] running on GCE node
I20260806 01:18:36.871940 19798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:36.872852 19798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:36.874096 19798 hybrid_clock.cc:648] HybridClock initialized: now 1785979116874046 us; error 68 us; skew 500 ppm
I20260806 01:18:36.874521 19798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:18:36 dist-test-slave-7nm7 krb5kdc[17842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979116, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:18:36.877897 19798 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:18:36.879415 19798 webserver.cc:460] Webserver started at http://127.25.236.130:34109/ using document root <none> and password file <none>
I20260806 01:18:36.880112 19798 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:36.880265 19798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:36.886057 19798 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:18:36.888855 19822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:36.890203 19798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:18:36.890466 19798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "b98987123d3e4758abbf3b5279a7d349"
format_stamp: "Formatted at 2026-08-06 01:17:50 on dist-test-slave-7nm7"
server_key: "614cb15279c875d08b76f639637d7f07"
server_key_iv: "81651d8e41f37bd5157a1d9974728562"
server_key_version: "encryptionkey@0"
I20260806 01:18:36.891470 19798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:36.911237 19798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:36.915808 19798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:36.916188 19798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:36.917317 19798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:18:36.918797 19798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:18:36.918932 19798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:36.919077 19798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:18:36.919175 19798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:36.955616 19798 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:43759
I20260806 01:18:36.955709 19983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:43759 every 8 connection(s)
I20260806 01:18:36.957074 19798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:18:36.961923 19989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:36.960162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:45279)
0806 01:18:36.960882 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:36.961130 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":277,"threads_started":1}
W20260806 01:18:36.962682 19985 heartbeater.cc:644] Failed to heartbeat to 127.25.236.190:35573 (0 consecutive failures): Network error: Failed to ping master at 127.25.236.190:35573: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
I20260806 01:18:36.962857 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 19798
I20260806 01:18:36.963567 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:43863
--local_ip_for_outbound_sockets=127.25.236.131
--tserver_master_addrs=127.25.236.190:35573
--webserver_port=38785
--webserver_interface=127.25.236.131
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:18:36.963721 19989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:36.963242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:45695)
0806 01:18:36.963391 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:36.963466 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 01:18:36.964836 19989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:36.964322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:36039)
0806 01:18:36.964534 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:36.964648 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260806 01:18:36.965256 19985 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:18:37.332508 19990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:37.332794 19990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:37.333000 19990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:37.340643 19990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:37.340749 19990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:37.340962 19990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:18:37.351543 19990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:43863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=38785
--tserver_master_addrs=127.25.236.190:35573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:37.352977 19990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:37.354377 19990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:37.368446 20000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:37.368480 19998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:37.368898 19997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:37.369144 19990 server_base.cc:1034] running on GCE node
I20260806 01:18:37.369943 19990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:37.370970 19990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:37.372165 19990 hybrid_clock.cc:648] HybridClock initialized: now 1785979117372111 us; error 56 us; skew 500 ppm
I20260806 01:18:37.372449 19990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:18:37 dist-test-slave-7nm7 krb5kdc[17842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979117, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:18:37.375972 19990 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:18:37.377913 19990 webserver.cc:460] Webserver started at http://127.25.236.131:38785/ using document root <none> and password file <none>
I20260806 01:18:37.378676 19990 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:37.378830 19990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:37.384227 19990 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 01:18:37.387238 20014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:37.388644 19990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:18:37.388860 19990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "e4ce8aa66b2b4b348b7bf5340446f02f"
format_stamp: "Formatted at 2026-08-06 01:17:50 on dist-test-slave-7nm7"
server_key: "bb813128446ec54169bd95e1a6fadc36"
server_key_iv: "2a24daa8a83d44e5a6974e9c509772f5"
server_key_version: "encryptionkey@0"
I20260806 01:18:37.389853 19990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:37.416728 19990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:37.422465 19990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:37.422767 19990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:37.423913 19990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:18:37.425433 19990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:18:37.425562 19990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:37.425714 19990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:18:37.425808 19990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:37.465751 19990 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:43863
I20260806 01:18:37.465816 20187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:43863 every 8 connection(s)
I20260806 01:18:37.467242 19990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:18:37.468531 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 19990
I20260806 01:18:37.471585 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:37.470005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:45191)
0806 01:18:37.470975 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:37.471243 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":522,"threads_started":1}
I20260806 01:18:37.473692 20197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:37.471506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:58749)
0806 01:18:37.472508 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:37.472769 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":343,"threads_started":1}
W20260806 01:18:37.474545 20188 heartbeater.cc:644] Failed to heartbeat to 127.25.236.190:35573 (0 consecutive failures): Network error: Failed to ping master at 127.25.236.190:35573: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
I20260806 01:18:37.475495 20197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:37.475158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:47479)
0806 01:18:37.475311 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:37.475389 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 01:18:37.476370 20197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:37.475996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:55501)
0806 01:18:37.476160 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:37.476258 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 01:18:37.476634 20188 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 01:18:37.966802 20200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:37.965894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:45535)
0806 01:18:37.966441 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:37.966555 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":247,"threads_started":1}
I20260806 01:18:38.473250 20202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:38.472557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:36293)
0806 01:18:38.473005 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:38.473086 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":213,"threads_started":1}
I20260806 01:18:38.477921 20203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:38.477132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:43075)
0806 01:18:38.477620 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:38.477705 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":267,"threads_started":1}
I20260806 01:18:38.968901 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:38.968020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:54631)
0806 01:18:38.968608 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:38.968722 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":294,"threads_started":1}
I20260806 01:18:39.474924 20205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:39.474174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:58179)
0806 01:18:39.474677 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:39.474759 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":285,"threads_started":1}
I20260806 01:18:39.479378 20206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:39.478744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:53445)
0806 01:18:39.479153 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:39.479228 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":195,"threads_started":1}
I20260806 01:18:39.970806 20208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:39.969955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:49171)
0806 01:18:39.970544 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:39.970636 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":364,"threads_started":1}
I20260806 01:18:40.478994 20209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:40.475798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:35243)
0806 01:18:40.478699 (+ 2901us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:40.478794 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2752,"thread_start_us":2620,"threads_started":1}
I20260806 01:18:40.480906 20210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:40.480183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:35943)
0806 01:18:40.480635 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:40.480723 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":262,"threads_started":1}
I20260806 01:18:40.972527 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:40.971821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:50343)
0806 01:18:40.972242 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:40.972332 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":195,"threads_started":1}
I20260806 01:18:41.480823 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:41.480039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:55863)
0806 01:18:41.480546 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:41.480641 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":264,"threads_started":1}
I20260806 01:18:41.482574 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:41.481714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:36951)
0806 01:18:41.482304 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:41.482398 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":219,"threads_started":1}
I20260806 01:18:41.974186 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:41.973462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:46483)
0806 01:18:41.973921 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:41.973999 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":246,"threads_started":1}
I20260806 01:18:42.482604 20218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:42.481744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:59403)
0806 01:18:42.482187 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:42.482346 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":216,"threads_started":1}
I20260806 01:18:42.486879 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:42.483613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:58745)
0806 01:18:42.486632 (+ 3019us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:42.486713 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2923,"thread_start_us":2813,"threads_started":1}
I20260806 01:18:42.975996 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:42.975223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:46451)
0806 01:18:42.975730 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:42.975828 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":278,"threads_started":1}
I20260806 01:18:43.484594 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:43.483811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:55047)
0806 01:18:43.484295 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:43.484375 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":223,"threads_started":1}
I20260806 01:18:43.488437 20225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:43.487717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:42489)
0806 01:18:43.488152 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:43.488288 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":220,"threads_started":1}
I20260806 01:18:43.978130 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:43.977137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:34445)
0806 01:18:43.977769 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:43.977866 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":270,"threads_started":1}
I20260806 01:18:44.486866 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:44.485860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:60341)
0806 01:18:44.486602 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:44.486700 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":253,"threads_started":1}
I20260806 01:18:44.490142 20228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:44.489390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:54047)
0806 01:18:44.489861 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:44.489939 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":220,"threads_started":1}
I20260806 01:18:44.979952 20229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:44.979126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:54309)
0806 01:18:44.979687 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:44.979772 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":320,"threads_started":1}
I20260806 01:18:45.492066 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:45.491292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:32955)
0806 01:18:45.491819 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:45.491902 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":277,"threads_started":1}
I20260806 01:18:45.502965 20231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:45.488054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:34849)
0806 01:18:45.502666 (+ 14612us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:45.502765 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14477,"thread_start_us":14667,"threads_started":1}
I20260806 01:18:45.981849 20233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:45.980926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:53801)
0806 01:18:45.981594 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:45.981677 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":260,"threads_started":1}
I20260806 01:18:46.494517 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:46.493497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:60905)
0806 01:18:46.494167 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:46.494317 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":400,"threads_started":1}
I20260806 01:18:46.505084 20236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:46.504165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:34371)
0806 01:18:46.504760 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:46.504871 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":310,"threads_started":1}
I20260806 01:18:46.984051 20237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:46.983028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:60747)
0806 01:18:46.983712 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:46.983834 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":362,"threads_started":1}
I20260806 01:18:47.496441 20238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:47.495551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:46515)
0806 01:18:47.496163 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:47.496252 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":365,"threads_started":1}
I20260806 01:18:47.507750 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:47.506724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:43715)
0806 01:18:47.507460 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:47.507557 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":438,"threads_started":1}
I20260806 01:18:47.986233 20241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:47.985334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:42519)
0806 01:18:47.985799 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:47.985896 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":236,"threads_started":1}
I20260806 01:18:48.498077 20243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:48.497355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:41095)
0806 01:18:48.497802 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:48.497899 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":228,"threads_started":1}
I20260806 01:18:48.509755 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:48.508920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:45113)
0806 01:18:48.509505 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:48.509589 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":350,"threads_started":1}
I20260806 01:18:48.988049 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:48.987377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:60033)
0806 01:18:48.987818 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:48.987895 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":220,"threads_started":1}
I20260806 01:18:49.499786 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:49.499057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:39139)
0806 01:18:49.499544 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:49.499629 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":257,"threads_started":1}
I20260806 01:18:49.511763 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:49.510861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:39059)
0806 01:18:49.511459 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:49.511581 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":339,"threads_started":1}
I20260806 01:18:49.990103 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:49.989253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:36279)
0806 01:18:49.989791 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:49.989890 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":285,"threads_started":1}
I20260806 01:18:50.502318 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:50.500984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:33773)
0806 01:18:50.501736 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:50.501869 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":428,"threads_started":1}
I20260806 01:18:50.513669 20251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:50.512730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:48963)
0806 01:18:50.513259 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:50.513458 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":232,"threads_started":1}
I20260806 01:18:50.992290 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:50.991497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:59949)
0806 01:18:50.991977 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:50.992079 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":244,"threads_started":1}
I20260806 01:18:51.506771 20254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:51.505649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:43309)
0806 01:18:51.506462 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:51.506569 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":353,"threads_started":1}
I20260806 01:18:51.516351 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:51.514991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:53509)
0806 01:18:51.515714 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:51.516044 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":368,"threads_started":1}
I20260806 01:18:51.994756 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:51.993748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.130:57391)
0806 01:18:51.994351 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:51.994461 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":253,"threads_started":1}
I20260806 01:18:52.477828 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 19551
I20260806 01:18:52.505661 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:52.479264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35268 (local address 127.25.236.129:41553)
0806 01:18:52.480121 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:52.480144 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:52.480203 (+ 59us) server_negotiation.cc:407] Connection header received
0806 01:18:52.482050 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:52.482061 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:52.482725 (+ 664us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:52.483453 (+ 728us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:52.485588 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:52.487426 (+ 1838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:52.489474 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:52.490638 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:52.490861 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:52.498033 (+ 7172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:52.498043 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:52.498063 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:52.498121 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:52.501981 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:52.502908 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:52.502916 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:52.502923 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:52.503099 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:52.503515 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:52.503521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:52.503524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:52.504100 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:52.504427 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:52.504903 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 01:18:52.505376 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":455,"threads_started":1}
I20260806 01:18:52.507797 19790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:18:52.510453 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:52.508649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.131:39665)
0806 01:18:52.509647 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:52.509833 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":644,"threads_started":1}
I20260806 01:18:52.601840 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:52.600661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:35573 (local address 127.25.236.129:45355)
0806 01:18:52.601243 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:52.601474 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.190:35573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":259,"threads_started":1}
I20260806 01:18:52.601830 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 19551
I20260806 01:18:52.629083 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 19798
I20260806 01:18:52.649791 20263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:52.630150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45854 (local address 127.25.236.130:43759)
0806 01:18:52.630790 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:52.630804 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:52.630843 (+ 39us) server_negotiation.cc:407] Connection header received
0806 01:18:52.632445 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:52.632455 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:52.632884 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:52.633376 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:52.635233 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:52.636529 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:52.637807 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:52.638588 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:52.638803 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:52.642529 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:52.642538 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:52.642554 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:52.642597 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:52.646101 (+ 3504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:52.647051 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:52.647058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:52.647066 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:52.647248 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:52.647641 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:52.647646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:52.647649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:52.648199 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:52.648531 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:52.649045 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 01:18:52.649488 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":346,"threads_started":1}
I20260806 01:18:52.651311 19975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:18:52.742527 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 19798
I20260806 01:18:52.774380 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 19990
I20260806 01:18:52.797104 20266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:52.775421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.25.236.131:43863)
0806 01:18:52.776031 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:52.776047 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:52.776089 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:18:52.777950 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:52.777960 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:52.778566 (+ 606us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:52.778872 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:52.780370 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:52.781670 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:52.783212 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:52.783898 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:52.784065 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:52.788185 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:52.788193 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:52.788223 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:52.788277 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:52.792589 (+ 4312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:52.793518 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:52.793527 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:52.793535 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:52.793728 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:52.794393 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:52.794398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:52.794400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:52.795007 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:52.795385 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:52.796380 (+ 995us) server_negotiation.cc:299] Negotiation successful
0806 01:18:52.796759 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":285,"threads_started":1}
I20260806 01:18:52.798682 20183 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:18:52.893504 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 19990
2026-08-06T01:18:52Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65967 ms)
[----------] 30 tests from SecurityITest (327472 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:18:53 dist-test-slave-7nm7 krb5kdc[20271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:18:53 dist-test-slave-7nm7 krb5kdc[20271](info): set up 2 sockets
Aug 06 01:18:53 dist-test-slave-7nm7 krb5kdc[20271](info): commencing operation
krb5kdc: starting...
W20260806 01:18:54.289917 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.345s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:18:54 dist-test-slave-7nm7 krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:18:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:18:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:18:54.637568 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:37295
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:44219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:37295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:18:54.985725 20287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:54.986049 20287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:54.986248 20287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:54.995023 20287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:18:54.995205 20287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:54.995325 20287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:54.995456 20287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:18:54.995534 20287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:18:55.007920 20287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:44219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:37295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:37295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:55.009387 20287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:55.010793 20287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:55.023979 20298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:55.024039 20302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:55.024065 20299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:55.025367 20287 server_base.cc:1034] running on GCE node
I20260806 01:18:55.026187 20287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:55.027665 20287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:55.028895 20287 hybrid_clock.cc:648] HybridClock initialized: now 1785979135028846 us; error 34 us; skew 500 ppm
I20260806 01:18:55.029362 20287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:18:55 dist-test-slave-7nm7 krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979135, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:18:55.032649 20287 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:18:55.034431 20287 webserver.cc:460] Webserver started at http://127.25.236.190:35863/ using document root <none> and password file <none>
I20260806 01:18:55.035323 20287 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:55.035506 20287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:55.035861 20287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:18:55.040056 20287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "0a1797f0f41e4ba0b877b91d12de872c"
format_stamp: "Formatted at 2026-08-06 01:18:55 on dist-test-slave-7nm7"
server_key: "f3ee50e986cd943f8f85eaa25a335254"
server_key_iv: "d151cf5ed13fd60d1f288216c6eade63"
server_key_version: "encryptionkey@0"
I20260806 01:18:55.040874 20287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "0a1797f0f41e4ba0b877b91d12de872c"
format_stamp: "Formatted at 2026-08-06 01:18:55 on dist-test-slave-7nm7"
server_key: "f3ee50e986cd943f8f85eaa25a335254"
server_key_iv: "d151cf5ed13fd60d1f288216c6eade63"
server_key_version: "encryptionkey@0"
I20260806 01:18:55.046579 20287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:18:55.051052 20309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:55.052605 20287 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 01:18:55.052809 20287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "0a1797f0f41e4ba0b877b91d12de872c"
format_stamp: "Formatted at 2026-08-06 01:18:55 on dist-test-slave-7nm7"
server_key: "f3ee50e986cd943f8f85eaa25a335254"
server_key_iv: "d151cf5ed13fd60d1f288216c6eade63"
server_key_version: "encryptionkey@0"
I20260806 01:18:55.053026 20287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:55.069186 20287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:55.073720 20287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:55.074039 20287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:55.094707 20287 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:37295
I20260806 01:18:55.094739 20392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:37295 every 8 connection(s)
I20260806 01:18:55.096125 20287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:18:55.100251 20393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:18:55.100629 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 20287
I20260806 01:18:55.100867 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:18:55.101732 26546 external_mini_cluster.cc:1428] Setting key d9c47ac3ace7be15a5afc0887019787e
I20260806 01:18:55.116091 20393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c: Bootstrap starting.
Aug 06 01:18:55 dist-test-slave-7nm7 krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:18:55.119903 20393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c: Neither blocks nor log segments found. Creating new log.
I20260806 01:18:55.121210 20393 log.cc:824] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:18:55.124298 20393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c: No bootstrap required, opened a new log
I20260806 01:18:55.127663 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:55.103909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38640 (local address 127.25.236.190:37295)
0806 01:18:55.104855 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:55.104867 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:55.104907 (+ 40us) server_negotiation.cc:407] Connection header received
0806 01:18:55.107168 (+ 2261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:55.107202 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:55.107594 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:55.107922 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:55.109914 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:55.111556 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:55.113531 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:55.114209 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:55.114427 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:55.119490 (+ 5063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:55.119498 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:55.119513 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:55.119553 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:55.123350 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:55.124102 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:55.124110 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:55.124116 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:55.124291 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:55.124695 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:55.124701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:55.124709 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:55.125231 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:55.125688 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:55.126204 (+ 516us) server_negotiation.cc:299] Negotiation successful
0806 01:18:55.126648 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":509,"threads_started":1}
I20260806 01:18:55.133775 20393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1797f0f41e4ba0b877b91d12de872c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37295 } }
I20260806 01:18:55.134277 20393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:18:55.134400 20393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1797f0f41e4ba0b877b91d12de872c, State: Initialized, Role: FOLLOWER
I20260806 01:18:55.135003 20393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1797f0f41e4ba0b877b91d12de872c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37295 } }
I20260806 01:18:55.135272 20393 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:18:55.135433 20393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:18:55.135617 20393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:18:55.137933 20393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1797f0f41e4ba0b877b91d12de872c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37295 } }
I20260806 01:18:55.138540 20393 leader_election.cc:302] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a1797f0f41e4ba0b877b91d12de872c; no voters:
I20260806 01:18:55.139192 20393 leader_election.cc:288] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:18:55.139344 20400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:18:55.140089 20400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [term 1 LEADER]: Becoming Leader. State: Replica: 0a1797f0f41e4ba0b877b91d12de872c, State: Running, Role: LEADER
I20260806 01:18:55.140724 20400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1797f0f41e4ba0b877b91d12de872c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37295 } }
I20260806 01:18:55.141435 20393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:18:55.143462 20403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a1797f0f41e4ba0b877b91d12de872c. Latest consensus state: current_term: 1 leader_uuid: "0a1797f0f41e4ba0b877b91d12de872c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1797f0f41e4ba0b877b91d12de872c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37295 } } }
I20260806 01:18:55.143878 20403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [sys.catalog]: This master's current role is: LEADER
I20260806 01:18:55.144443 20402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a1797f0f41e4ba0b877b91d12de872c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1797f0f41e4ba0b877b91d12de872c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37295 } } }
I20260806 01:18:55.144917 20402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c [sys.catalog]: This master's current role is: LEADER
I20260806 01:18:55.148759 20406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:18:55.155844 20406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:18:55.167196 20406 catalog_manager.cc:1347] Generated new cluster ID: 7958efdcfcad4667bbd291e26eb73552
I20260806 01:18:55.167344 20406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:18:55.187227 20406 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:18:55.189049 20406 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:18:55.196925 20406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c: Generated new TSK 0
I20260806 01:18:55.197836 20406 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:18:55.295892 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:37295
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:44219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:18:55.681095 20431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:55.681372 20431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:55.681533 20431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:55.688725 20431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:55.688838 20431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:55.689035 20431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:18:55.699623 20431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:44219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:37295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:55.701009 20431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:55.702368 20431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:55.716252 20443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:55.716313 20449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:55.716799 20442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:55.717514 20431 server_base.cc:1034] running on GCE node
I20260806 01:18:55.718189 20431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:55.719435 20431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:55.720693 20431 hybrid_clock.cc:648] HybridClock initialized: now 1785979135720655 us; error 48 us; skew 500 ppm
I20260806 01:18:55.721026 20431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:18:55 dist-test-slave-7nm7 krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979135, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:18:55.725129 20431 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:18:55.726820 20431 webserver.cc:460] Webserver started at http://127.25.236.129:44831/ using document root <none> and password file <none>
I20260806 01:18:55.727525 20431 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:55.727677 20431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:55.728000 20431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:18:55.731163 20431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "53c5ff48ca784062a62cca1be2d1587a"
format_stamp: "Formatted at 2026-08-06 01:18:55 on dist-test-slave-7nm7"
server_key: "3b41496ecf2815d42666017bde1898da"
server_key_iv: "67000353f399bc281fbe3d1813503e5f"
server_key_version: "encryptionkey@0"
I20260806 01:18:55.731952 20431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "53c5ff48ca784062a62cca1be2d1587a"
format_stamp: "Formatted at 2026-08-06 01:18:55 on dist-test-slave-7nm7"
server_key: "3b41496ecf2815d42666017bde1898da"
server_key_iv: "67000353f399bc281fbe3d1813503e5f"
server_key_version: "encryptionkey@0"
I20260806 01:18:55.737259 20431 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:18:55.740767 20460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:55.742452 20431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:18:55.742622 20431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "53c5ff48ca784062a62cca1be2d1587a"
format_stamp: "Formatted at 2026-08-06 01:18:55 on dist-test-slave-7nm7"
server_key: "3b41496ecf2815d42666017bde1898da"
server_key_iv: "67000353f399bc281fbe3d1813503e5f"
server_key_version: "encryptionkey@0"
I20260806 01:18:55.742820 20431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:55.759137 20431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:55.764199 20431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:55.764506 20431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:55.765645 20431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:18:55.767122 20431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:18:55.767199 20431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:55.767329 20431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:18:55.767395 20431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:55.807772 20431 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:45289
I20260806 01:18:55.807816 20639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:45289 every 8 connection(s)
I20260806 01:18:55.809129 20431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:18:55.816558 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 20431
I20260806 01:18:55.816802 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:18:55.817615 26546 external_mini_cluster.cc:1428] Setting key 116b6344e5023ffe0c4c2b51f432b2f0
Aug 06 01:18:55 dist-test-slave-7nm7 krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979135, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:18:55.833958 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:55.812087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:46795 (local address 127.25.236.190:37295)
0806 01:18:55.812684 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:55.812689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:55.813747 (+ 1058us) server_negotiation.cc:407] Connection header received
0806 01:18:55.815832 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:55.815838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:55.815931 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:55.816070 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:55.817703 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:55.818690 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:55.820371 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:55.820775 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:55.820916 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:55.826375 (+ 5459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:55.826384 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:55.826392 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:55.826438 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:55.830296 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:55.831282 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:55.831286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:55.831289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:55.831412 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:55.831980 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:55.831984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:55.831986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:55.832333 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:55.832478 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:55.833478 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0806 01:18:55.833682 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":261,"threads_started":1}
I20260806 01:18:55.836037 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:55.812733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:37295 (local address 127.25.236.129:46795)
0806 01:18:55.813570 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:55.813620 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:55.815543 (+ 1923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:55.816298 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:55.816306 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:55.816942 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:18:55.817512 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:55.817539 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:55.818850 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:55.818854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:55.820220 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:55.820232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:55.821010 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:55.821020 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:55.821256 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:55.822736 (+ 1480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:18:55.822763 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:18:55.826010 (+ 3247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:18:55.830473 (+ 4463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:18:55.830481 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:18:55.830496 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:18:55.830969 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:18:55.831537 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:18:55.831545 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:18:55.831551 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:18:55.831834 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:18:55.832474 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:18:55.832486 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:18:55.833021 (+ 535us) client_negotiation.cc:769] Sending connection context
0806 01:18:55.833355 (+ 334us) client_negotiation.cc:240] Negotiation successful
0806 01:18:55.833986 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":388,"threads_started":1}
I20260806 01:18:55.837806 20640 heartbeater.cc:347] Connected to a master server at 127.25.236.190:37295
I20260806 01:18:55.838182 20640 heartbeater.cc:464] Registering TS with master...
I20260806 01:18:55.839123 20640 heartbeater.cc:511] Master 127.25.236.190:37295 requested a full tablet report, sending...
I20260806 01:18:55.841673 20345 ts_manager.cc:194] Registered new tserver with Master: 53c5ff48ca784062a62cca1be2d1587a (127.25.236.129:45289)
I20260806 01:18:55.843365 20345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:46795
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:18:55.898604 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:37295
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:44219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:18:56.277527 20647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:56.277877 20647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:56.278048 20647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:56.285763 20647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:56.285926 20647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:56.286100 20647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:18:56.296286 20647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:44219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:37295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:56.297668 20647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:56.299034 20647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:56.313365 20653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:56.313937 20655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:56.314517 20658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:56.315222 20647 server_base.cc:1034] running on GCE node
I20260806 01:18:56.315888 20647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:56.316905 20647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:56.318185 20647 hybrid_clock.cc:648] HybridClock initialized: now 1785979136318116 us; error 87 us; skew 500 ppm
I20260806 01:18:56.318576 20647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:18:56 dist-test-slave-7nm7 krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979136, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:18:56.321815 20647 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:18:56.323386 20647 webserver.cc:460] Webserver started at http://127.25.236.130:45565/ using document root <none> and password file <none>
I20260806 01:18:56.324059 20647 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:56.324301 20647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:56.324620 20647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:18:56.328141 20647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "f58e9a6107034e1db609d41a12796367"
format_stamp: "Formatted at 2026-08-06 01:18:56 on dist-test-slave-7nm7"
server_key: "91c78f030eda22125dbc36004a7b41cf"
server_key_iv: "fa4fbbdf2c2dd006e624229be17e7efe"
server_key_version: "encryptionkey@0"
I20260806 01:18:56.328897 20647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "f58e9a6107034e1db609d41a12796367"
format_stamp: "Formatted at 2026-08-06 01:18:56 on dist-test-slave-7nm7"
server_key: "91c78f030eda22125dbc36004a7b41cf"
server_key_iv: "fa4fbbdf2c2dd006e624229be17e7efe"
server_key_version: "encryptionkey@0"
I20260806 01:18:56.335237 20647 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 01:18:56.338948 20668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:56.340278 20647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:18:56.340426 20647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "f58e9a6107034e1db609d41a12796367"
format_stamp: "Formatted at 2026-08-06 01:18:56 on dist-test-slave-7nm7"
server_key: "91c78f030eda22125dbc36004a7b41cf"
server_key_iv: "fa4fbbdf2c2dd006e624229be17e7efe"
server_key_version: "encryptionkey@0"
I20260806 01:18:56.340606 20647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:56.385303 20647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:56.391206 20647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:56.391501 20647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:56.392509 20647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:18:56.394192 20647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:18:56.394366 20647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 01:18:56.394479 20647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:18:56.394529 20647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:56.436801 20647 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:45369
I20260806 01:18:56.436874 20842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:45369 every 8 connection(s)
I20260806 01:18:56.438190 20647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:18:56.446800 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 20647
I20260806 01:18:56.447187 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:18:56.447925 26546 external_mini_cluster.cc:1428] Setting key bbeda52924f0083877961c2a60516be5
Aug 06 01:18:56 dist-test-slave-7nm7 krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979136, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:18:56.469574 20845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:56.445044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:60963 (local address 127.25.236.190:37295)
0806 01:18:56.445713 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:56.445722 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:56.447694 (+ 1972us) server_negotiation.cc:407] Connection header received
0806 01:18:56.450083 (+ 2389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:56.450087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:56.450178 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:56.450311 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:56.453001 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:56.453737 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:56.456621 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:56.457145 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:56.457302 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:56.461923 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:56.461929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:56.461932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:56.461966 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:56.465897 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:56.466904 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:56.466908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:56.466911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:56.467062 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:56.467611 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:56.467614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:56.467616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:56.467903 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:56.468023 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:56.468926 (+ 903us) server_negotiation.cc:299] Negotiation successful
0806 01:18:56.469194 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":326,"threads_started":1}
I20260806 01:18:56.470733 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:56.445593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:37295 (local address 127.25.236.130:60963)
0806 01:18:56.447000 (+ 1407us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:56.447052 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:56.449805 (+ 2753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:56.450599 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:56.450610 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:56.451470 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:18:56.452591 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:56.452651 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:56.453946 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:56.453955 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:56.456365 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:56.456386 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:56.457423 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:56.457433 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:56.457889 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:56.458826 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:18:56.458863 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:18:56.461664 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:18:56.466101 (+ 4437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:18:56.466112 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:18:56.466132 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:18:56.466736 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:18:56.467185 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:18:56.467192 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:18:56.467198 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:18:56.467476 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:18:56.468042 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:18:56.468052 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:18:56.468510 (+ 458us) client_negotiation.cc:769] Sending connection context
0806 01:18:56.468808 (+ 298us) client_negotiation.cc:240] Negotiation successful
0806 01:18:56.469401 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"thread_start_us":437,"threads_started":1}
I20260806 01:18:56.472538 20843 heartbeater.cc:347] Connected to a master server at 127.25.236.190:37295
I20260806 01:18:56.472903 20843 heartbeater.cc:464] Registering TS with master...
I20260806 01:18:56.474292 20843 heartbeater.cc:511] Master 127.25.236.190:37295 requested a full tablet report, sending...
I20260806 01:18:56.475953 20345 ts_manager.cc:194] Registered new tserver with Master: f58e9a6107034e1db609d41a12796367 (127.25.236.130:45369)
I20260806 01:18:56.477267 20345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:60963
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:18:56.526590 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:37295
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:44219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:18:56.847802 20640 heartbeater.cc:503] Master 127.25.236.190:37295 was elected leader, sending a full tablet report...
W20260806 01:18:56.900715 20854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:56.901018 20854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:56.901243 20854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:56.911893 20854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:56.912019 20854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:56.912142 20854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:18:56.924629 20854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:44219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:37295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:56.926412 20854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:56.927802 20854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:56.941614 20860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:56.941709 20861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:56.941749 20864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:56.942692 20854 server_base.cc:1034] running on GCE node
I20260806 01:18:56.943356 20854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:56.944481 20854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:56.945694 20854 hybrid_clock.cc:648] HybridClock initialized: now 1785979136945654 us; error 30 us; skew 500 ppm
I20260806 01:18:56.946130 20854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:18:56 dist-test-slave-7nm7 krb5kdc[20271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979136, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:18:56.949542 20854 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:18:56.951414 20854 webserver.cc:460] Webserver started at http://127.25.236.131:33037/ using document root <none> and password file <none>
I20260806 01:18:56.952083 20854 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:56.952190 20854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:56.952454 20854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:18:56.956144 20854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "7cf1668566d748ed8321d053ba0ff840"
format_stamp: "Formatted at 2026-08-06 01:18:56 on dist-test-slave-7nm7"
server_key: "a4d3e20915e1647a5d8a1b76ad0593c0"
server_key_iv: "4d1c9166d3821cc46eba2f42908f58df"
server_key_version: "encryptionkey@0"
I20260806 01:18:56.957175 20854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "7cf1668566d748ed8321d053ba0ff840"
format_stamp: "Formatted at 2026-08-06 01:18:56 on dist-test-slave-7nm7"
server_key: "a4d3e20915e1647a5d8a1b76ad0593c0"
server_key_iv: "4d1c9166d3821cc46eba2f42908f58df"
server_key_version: "encryptionkey@0"
I20260806 01:18:56.963315 20854 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260806 01:18:56.966888 20872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:56.968284 20854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:18:56.968468 20854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "7cf1668566d748ed8321d053ba0ff840"
format_stamp: "Formatted at 2026-08-06 01:18:56 on dist-test-slave-7nm7"
server_key: "a4d3e20915e1647a5d8a1b76ad0593c0"
server_key_iv: "4d1c9166d3821cc46eba2f42908f58df"
server_key_version: "encryptionkey@0"
I20260806 01:18:56.968686 20854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:56.991518 20854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:56.996235 20854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:56.996569 20854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:56.997934 20854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:18:56.999552 20854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:18:56.999642 20854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:56.999833 20854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:18:56.999892 20854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:57.036041 20854 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:37265
I20260806 01:18:57.036118 21046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:37265 every 8 connection(s)
I20260806 01:18:57.037341 20854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:18:57.042068 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 20854
I20260806 01:18:57.042315 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:18:57.042881 26546 external_mini_cluster.cc:1428] Setting key 8ef9c8233fcb4e5077a0315c872fb9ea
Aug 06 01:18:57 dist-test-slave-7nm7 krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979136, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:18:57.058773 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.039994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:37591 (local address 127.25.236.190:37295)
0806 01:18:57.040538 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:57.040545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:57.041628 (+ 1083us) server_negotiation.cc:407] Connection header received
0806 01:18:57.044184 (+ 2556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:57.044192 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:57.044283 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:57.044392 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:57.046090 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.047030 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.048593 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.048993 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.049094 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.052650 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:57.052663 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:57.052666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:57.052697 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:57.055359 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:57.056022 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:57.056025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:57.056027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:57.056197 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:57.056659 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:57.056664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:57.056667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:57.057021 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:57.057176 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:57.058071 (+ 895us) server_negotiation.cc:299] Negotiation successful
0806 01:18:57.058352 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":302,"threads_started":1}
I20260806 01:18:57.059667 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.040663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:37295 (local address 127.25.236.131:37591)
0806 01:18:57.041453 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:57.041513 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:57.043823 (+ 2310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:57.044597 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:57.044604 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:57.045288 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:18:57.045892 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:57.045932 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.047160 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.047164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:57.048461 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:57.048473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.049193 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.049198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:57.049387 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.050094 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:18:57.050126 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:18:57.052452 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:18:57.055487 (+ 3035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:18:57.055494 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:18:57.055505 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:18:57.055922 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:18:57.056300 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:18:57.056303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:18:57.056305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:18:57.056534 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:18:57.057170 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:18:57.057179 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:18:57.057635 (+ 456us) client_negotiation.cc:769] Sending connection context
0806 01:18:57.057950 (+ 315us) client_negotiation.cc:240] Negotiation successful
0806 01:18:57.058480 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":342,"threads_started":1}
I20260806 01:18:57.061071 21049 heartbeater.cc:347] Connected to a master server at 127.25.236.190:37295
I20260806 01:18:57.061457 21049 heartbeater.cc:464] Registering TS with master...
I20260806 01:18:57.062157 21049 heartbeater.cc:511] Master 127.25.236.190:37295 requested a full tablet report, sending...
I20260806 01:18:57.063602 20345 ts_manager.cc:194] Registered new tserver with Master: 7cf1668566d748ed8321d053ba0ff840 (127.25.236.131:37265)
I20260806 01:18:57.064390 20345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:37591
I20260806 01:18:57.070936 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:18:57.088114 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.075801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38644 (local address 127.25.236.190:37295)
0806 01:18:57.075985 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:57.075992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:57.076466 (+ 474us) server_negotiation.cc:407] Connection header received
0806 01:18:57.076581 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:57.076584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:57.076650 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:57.076728 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:57.078177 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.078926 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.080227 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.080701 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.080823 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.083189 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:57.083194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:57.083197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:57.083226 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:57.085636 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:57.086275 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:57.086279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:57.086281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:57.086405 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:57.086741 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:57.086744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:57.086746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:57.087051 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:57.087225 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:57.087591 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 01:18:57.087755 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:18:57.092154 20345 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:18:57.099337 20345 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:18:57.122886 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.111430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:45369 (local address 127.0.0.1:60836)
0806 01:18:57.112881 (+ 1451us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:57.112937 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:57.113130 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:57.115400 (+ 2270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:57.115409 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:57.115449 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:18:57.115869 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:57.115883 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.117903 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.117909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:57.120278 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:57.120291 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.121845 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.121851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:57.122032 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.122084 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 01:18:57.122213 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 01:18:57.122424 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1162,"mutex_wait_us":62,"thread_start_us":1010,"threads_started":1}
I20260806 01:18:57.123847 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.111543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60836 (local address 127.25.236.130:45369)
0806 01:18:57.114749 (+ 3206us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:57.114757 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:57.114784 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:18:57.114881 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:57.114889 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:57.115150 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:57.116045 (+ 895us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:18:57.116276 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.117723 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.120439 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.121644 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.121815 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.121988 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:57.123381 (+ 1393us) server_negotiation.cc:299] Negotiation successful
0806 01:18:57.123562 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2708,"thread_start_us":2590,"threads_started":1}
I20260806 01:18:57.123943 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.111684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:45289 (local address 127.0.0.1:57120)
0806 01:18:57.114769 (+ 3085us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:57.114792 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:57.115030 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:57.116058 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:57.116066 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:57.116113 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:18:57.116613 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:57.116625 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.118474 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.118479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:57.120840 (+ 2361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:57.120855 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.123089 (+ 2234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.123095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:57.123275 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.123307 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:18:57.123401 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 01:18:57.123539 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2918,"mutex_wait_us":125,"thread_start_us":2298,"threads_started":1}
I20260806 01:18:57.124693 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.113891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:37265 (local address 127.0.0.1:46980)
0806 01:18:57.115600 (+ 1709us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:57.115633 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:57.116155 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:57.117332 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:57.117337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:57.117361 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:18:57.117659 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:57.117670 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.119617 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.119623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:57.121512 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:57.121523 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.123571 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.123577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:57.123761 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.123799 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 01:18:57.123910 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 01:18:57.124321 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1574,"mutex_wait_us":50,"thread_start_us":538,"threads_started":1}
I20260806 01:18:57.124707 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.112354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57120 (local address 127.25.236.129:45289)
0806 01:18:57.113801 (+ 1447us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:57.113813 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:57.114905 (+ 1092us) server_negotiation.cc:407] Connection header received
0806 01:18:57.115263 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:57.115274 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:57.115522 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:57.115709 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:18:57.116858 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.118317 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.121006 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.122562 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.122768 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.122932 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:57.124237 (+ 1305us) server_negotiation.cc:299] Negotiation successful
0806 01:18:57.124392 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1250,"thread_start_us":1105,"threads_started":1}
I20260806 01:18:57.125243 21077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.114462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46980 (local address 127.25.236.131:37265)
0806 01:18:57.116821 (+ 2359us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:57.116828 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:57.116856 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:18:57.116939 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:57.116945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:57.117131 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:57.117321 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:18:57.117868 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.119462 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.121661 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.123317 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.123539 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.123689 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:57.123960 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 01:18:57.124933 (+ 973us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2193,"thread_start_us":2046,"threads_started":1}
I20260806 01:18:57.127345 20740 tablet_service.cc:1467] Processing CreateTablet for tablet 314467bd7d0f4fa5abbcded0d57b494c (DEFAULT_TABLE table=test-table [id=b0ee08d400db488fb1b36f7b38a416d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:18:57.127997 20538 tablet_service.cc:1467] Processing CreateTablet for tablet 314467bd7d0f4fa5abbcded0d57b494c (DEFAULT_TABLE table=test-table [id=b0ee08d400db488fb1b36f7b38a416d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:18:57.128789 20740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 314467bd7d0f4fa5abbcded0d57b494c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:18:57.128525 20943 tablet_service.cc:1467] Processing CreateTablet for tablet 314467bd7d0f4fa5abbcded0d57b494c (DEFAULT_TABLE table=test-table [id=b0ee08d400db488fb1b36f7b38a416d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:18:57.129325 20538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 314467bd7d0f4fa5abbcded0d57b494c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:18:57.129823 20943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 314467bd7d0f4fa5abbcded0d57b494c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:18:57.140120 21078 tablet_bootstrap.cc:492] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367: Bootstrap starting.
I20260806 01:18:57.144294 21080 tablet_bootstrap.cc:492] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840: Bootstrap starting.
I20260806 01:18:57.145195 21078 tablet_bootstrap.cc:654] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367: Neither blocks nor log segments found. Creating new log.
I20260806 01:18:57.145267 21079 tablet_bootstrap.cc:492] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a: Bootstrap starting.
I20260806 01:18:57.146762 21078 log.cc:824] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367: Log is configured to *not* fsync() on all Append() calls
I20260806 01:18:57.148829 21079 tablet_bootstrap.cc:654] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a: Neither blocks nor log segments found. Creating new log.
I20260806 01:18:57.148999 21080 tablet_bootstrap.cc:654] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840: Neither blocks nor log segments found. Creating new log.
I20260806 01:18:57.150350 21080 log.cc:824] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840: Log is configured to *not* fsync() on all Append() calls
I20260806 01:18:57.150363 21079 log.cc:824] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:18:57.150827 21078 tablet_bootstrap.cc:492] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367: No bootstrap required, opened a new log
I20260806 01:18:57.151188 21078 ts_tablet_manager.cc:1397] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260806 01:18:57.156924 21080 tablet_bootstrap.cc:492] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840: No bootstrap required, opened a new log
I20260806 01:18:57.157234 21080 ts_tablet_manager.cc:1397] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260806 01:18:57.157603 21079 tablet_bootstrap.cc:492] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a: No bootstrap required, opened a new log
I20260806 01:18:57.157852 21079 ts_tablet_manager.cc:1397] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260806 01:18:57.163393 21078 raft_consensus.cc:348] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58e9a6107034e1db609d41a12796367" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 45369 } } peers { permanent_uuid: "53c5ff48ca784062a62cca1be2d1587a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45289 } } peers { permanent_uuid: "7cf1668566d748ed8321d053ba0ff840" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 37265 } }
I20260806 01:18:57.163879 21078 raft_consensus.cc:374] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:18:57.163978 21078 raft_consensus.cc:729] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f58e9a6107034e1db609d41a12796367, State: Initialized, Role: FOLLOWER
I20260806 01:18:57.164628 21078 consensus_queue.cc:260] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58e9a6107034e1db609d41a12796367" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 45369 } } peers { permanent_uuid: "53c5ff48ca784062a62cca1be2d1587a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45289 } } peers { permanent_uuid: "7cf1668566d748ed8321d053ba0ff840" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 37265 } }
I20260806 01:18:57.165804 20843 heartbeater.cc:503] Master 127.25.236.190:37295 was elected leader, sending a full tablet report...
I20260806 01:18:57.165706 21080 raft_consensus.cc:348] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58e9a6107034e1db609d41a12796367" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 45369 } } peers { permanent_uuid: "53c5ff48ca784062a62cca1be2d1587a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45289 } } peers { permanent_uuid: "7cf1668566d748ed8321d053ba0ff840" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 37265 } }
I20260806 01:18:57.166111 21080 raft_consensus.cc:374] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:18:57.166217 21080 raft_consensus.cc:729] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cf1668566d748ed8321d053ba0ff840, State: Initialized, Role: FOLLOWER
I20260806 01:18:57.166501 21078 ts_tablet_manager.cc:1428] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260806 01:18:57.166899 21080 consensus_queue.cc:260] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58e9a6107034e1db609d41a12796367" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 45369 } } peers { permanent_uuid: "53c5ff48ca784062a62cca1be2d1587a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45289 } } peers { permanent_uuid: "7cf1668566d748ed8321d053ba0ff840" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 37265 } }
I20260806 01:18:57.167943 21049 heartbeater.cc:503] Master 127.25.236.190:37295 was elected leader, sending a full tablet report...
I20260806 01:18:57.168538 21080 ts_tablet_manager.cc:1428] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260806 01:18:57.168337 21079 raft_consensus.cc:348] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58e9a6107034e1db609d41a12796367" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 45369 } } peers { permanent_uuid: "53c5ff48ca784062a62cca1be2d1587a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45289 } } peers { permanent_uuid: "7cf1668566d748ed8321d053ba0ff840" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 37265 } }
I20260806 01:18:57.168710 21079 raft_consensus.cc:374] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:18:57.168809 21079 raft_consensus.cc:729] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53c5ff48ca784062a62cca1be2d1587a, State: Initialized, Role: FOLLOWER
I20260806 01:18:57.169349 21079 consensus_queue.cc:260] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58e9a6107034e1db609d41a12796367" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 45369 } } peers { permanent_uuid: "53c5ff48ca784062a62cca1be2d1587a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45289 } } peers { permanent_uuid: "7cf1668566d748ed8321d053ba0ff840" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 37265 } }
I20260806 01:18:57.171092 21079 ts_tablet_manager.cc:1428] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
W20260806 01:18:57.194088 20844 tablet.cc:2369] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:18:57.269315 21089 raft_consensus.cc:482] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:18:57.269685 21089 raft_consensus.cc:504] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58e9a6107034e1db609d41a12796367" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 45369 } } peers { permanent_uuid: "53c5ff48ca784062a62cca1be2d1587a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45289 } } peers { permanent_uuid: "7cf1668566d748ed8321d053ba0ff840" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 37265 } }
I20260806 01:18:57.271843 21089 leader_election.cc:288] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f58e9a6107034e1db609d41a12796367 (127.25.236.130:45369), 7cf1668566d748ed8321d053ba0ff840 (127.25.236.131:37265)
I20260806 01:18:57.279417 21077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.272478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:55753 (local address 127.25.236.131:37265)
0806 01:18:57.272729 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:57.272737 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:57.273737 (+ 1000us) server_negotiation.cc:407] Connection header received
0806 01:18:57.274011 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:57.274016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:57.274096 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:57.274199 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:18:57.274845 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.275595 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.277526 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.278410 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.278593 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.278683 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:57.279009 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 01:18:57.279151 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:18:57.279475 21092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.272400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:37265 (local address 127.25.236.129:55753)
0806 01:18:57.273592 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:57.273612 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:57.273852 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:57.274217 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:57.274222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:57.274278 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:18:57.274693 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:57.274704 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.275734 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.275741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:57.277357 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:57.277370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.278653 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.278660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:57.278833 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.278859 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 01:18:57.278970 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 01:18:57.279110 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036,"threads_started":1}
I20260806 01:18:57.281471 20979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "314467bd7d0f4fa5abbcded0d57b494c" candidate_uuid: "53c5ff48ca784062a62cca1be2d1587a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cf1668566d748ed8321d053ba0ff840" is_pre_election: true
I20260806 01:18:57.282155 20979 raft_consensus.cc:2454] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53c5ff48ca784062a62cca1be2d1587a in term 0.
I20260806 01:18:57.282964 20461 leader_election.cc:302] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53c5ff48ca784062a62cca1be2d1587a, 7cf1668566d748ed8321d053ba0ff840; no voters:
I20260806 01:18:57.283504 21089 raft_consensus.cc:2781] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:18:57.283692 21089 raft_consensus.cc:482] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:18:57.283828 21089 raft_consensus.cc:3037] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:18:57.285595 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.272852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:41765 (local address 127.25.236.130:45369)
0806 01:18:57.273230 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:57.273242 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:57.279832 (+ 6590us) server_negotiation.cc:407] Connection header received
0806 01:18:57.280022 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:57.280028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:57.280149 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:57.280251 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:18:57.280653 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.281660 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.283736 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.284657 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.284811 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.284866 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:57.285281 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 01:18:57.285397 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 01:18:57.285687 21092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.272512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:45369 (local address 127.25.236.129:41765)
0806 01:18:57.279705 (+ 7193us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:57.279732 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:57.279874 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:57.280273 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:57.280278 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:57.280309 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:18:57.280473 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:57.280482 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.281897 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.281903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:57.283515 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:57.283533 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.284853 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.284859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:57.285053 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.285094 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 01:18:57.285219 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 01:18:57.285372 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7071,"thread_start_us":767,"threads_started":1}
I20260806 01:18:57.286835 21089 raft_consensus.cc:504] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58e9a6107034e1db609d41a12796367" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 45369 } } peers { permanent_uuid: "53c5ff48ca784062a62cca1be2d1587a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45289 } } peers { permanent_uuid: "7cf1668566d748ed8321d053ba0ff840" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 37265 } }
I20260806 01:18:57.286819 20772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "314467bd7d0f4fa5abbcded0d57b494c" candidate_uuid: "53c5ff48ca784062a62cca1be2d1587a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f58e9a6107034e1db609d41a12796367" is_pre_election: true
I20260806 01:18:57.287326 20772 raft_consensus.cc:2454] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53c5ff48ca784062a62cca1be2d1587a in term 0.
I20260806 01:18:57.287570 21089 leader_election.cc:288] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [CANDIDATE]: Term 1 election: Requested vote from peers f58e9a6107034e1db609d41a12796367 (127.25.236.130:45369), 7cf1668566d748ed8321d053ba0ff840 (127.25.236.131:37265)
I20260806 01:18:57.288008 20772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "314467bd7d0f4fa5abbcded0d57b494c" candidate_uuid: "53c5ff48ca784062a62cca1be2d1587a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f58e9a6107034e1db609d41a12796367"
I20260806 01:18:57.288215 20772 raft_consensus.cc:3037] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367 [term 0 FOLLOWER]: Advancing to term 1
W20260806 01:18:57.289197 21050 tablet.cc:2369] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:18:57.290503 20979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "314467bd7d0f4fa5abbcded0d57b494c" candidate_uuid: "53c5ff48ca784062a62cca1be2d1587a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cf1668566d748ed8321d053ba0ff840"
I20260806 01:18:57.290735 20979 raft_consensus.cc:3037] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:18:57.291496 20772 raft_consensus.cc:2454] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53c5ff48ca784062a62cca1be2d1587a in term 1.
I20260806 01:18:57.292371 20465 leader_election.cc:302] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53c5ff48ca784062a62cca1be2d1587a, f58e9a6107034e1db609d41a12796367; no voters:
I20260806 01:18:57.292784 21089 raft_consensus.cc:2781] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:18:57.293370 21089 raft_consensus.cc:686] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [term 1 LEADER]: Becoming Leader. State: Replica: 53c5ff48ca784062a62cca1be2d1587a, State: Running, Role: LEADER
I20260806 01:18:57.293947 21089 consensus_queue.cc:237] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58e9a6107034e1db609d41a12796367" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 45369 } } peers { permanent_uuid: "53c5ff48ca784062a62cca1be2d1587a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45289 } } peers { permanent_uuid: "7cf1668566d748ed8321d053ba0ff840" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 37265 } }
I20260806 01:18:57.294458 20979 raft_consensus.cc:2454] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53c5ff48ca784062a62cca1be2d1587a in term 1.
I20260806 01:18:57.301704 20340 catalog_manager.cc:5495] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a reported cstate change: term changed from 0 to 1, leader changed from <none> to 53c5ff48ca784062a62cca1be2d1587a (127.25.236.129). New cstate: current_term: 1 leader_uuid: "53c5ff48ca784062a62cca1be2d1587a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58e9a6107034e1db609d41a12796367" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 45369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53c5ff48ca784062a62cca1be2d1587a" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cf1668566d748ed8321d053ba0ff840" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 37265 } health_report { overall_health: UNKNOWN } } }
W20260806 01:18:57.311857 20641 tablet.cc:2369] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:18:57.367290 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.360454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57126 (local address 127.25.236.129:45289)
0806 01:18:57.360734 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:57.360740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:57.360765 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:18:57.360905 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:57.360909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:57.361028 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:57.361204 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:18:57.361714 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.362912 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.364428 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.364874 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.364989 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.365835 (+ 846us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:18:57.366586 (+ 751us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:18:57.366725 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:57.366942 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 01:18:57.367022 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 01:18:57.377027 20772 raft_consensus.cc:1261] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367 [term 1 FOLLOWER]: Refusing update from remote peer 53c5ff48ca784062a62cca1be2d1587a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:18:57.377385 20979 raft_consensus.cc:1261] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840 [term 1 FOLLOWER]: Refusing update from remote peer 53c5ff48ca784062a62cca1be2d1587a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:18:57.378160 21093 consensus_queue.cc:1035] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f58e9a6107034e1db609d41a12796367" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 45369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:18:57.378854 21089 consensus_queue.cc:1035] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cf1668566d748ed8321d053ba0ff840" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 37265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:18:57.392011 21097 mvcc.cc:205] Tried to move back new op lower bound from 7315370546684821504 to 7315370546362048512. Current Snapshot: MvccSnapshot[applied={T|T < 7315370546684821504}]
I20260806 01:18:57.409574 20340 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:18:57.409901 20340 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:18:57.414429 20340 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a1797f0f41e4ba0b877b91d12de872c: Sending DeleteTablet for 3 replicas of tablet 314467bd7d0f4fa5abbcded0d57b494c
I20260806 01:18:57.415892 20740 tablet_service.cc:1514] Processing DeleteTablet for tablet 314467bd7d0f4fa5abbcded0d57b494c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:18:57 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:60836
I20260806 01:18:57.416265 20538 tablet_service.cc:1514] Processing DeleteTablet for tablet 314467bd7d0f4fa5abbcded0d57b494c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:18:57 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:57120
I20260806 01:18:57.416538 20943 tablet_service.cc:1514] Processing DeleteTablet for tablet 314467bd7d0f4fa5abbcded0d57b494c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:18:57 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:46980
I20260806 01:18:57.422811 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 20431
I20260806 01:18:57.426818 21112 tablet_replica.cc:330] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840: stopping tablet replica
I20260806 01:18:57.427819 21110 tablet_replica.cc:330] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367: stopping tablet replica
I20260806 01:18:57.428087 21112 raft_consensus.cc:2229] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:18:57.428218 21110 raft_consensus.cc:2229] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:18:57.428755 21112 raft_consensus.cc:2258] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:18:57.428755 21110 raft_consensus.cc:2258] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:18:57.429656 21111 tablet_replica.cc:330] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a: stopping tablet replica
I20260806 01:18:57.430986 21110 ts_tablet_manager.cc:1905] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:18:57.431510 21111 raft_consensus.cc:2229] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:18:57.432914 21112 ts_tablet_manager.cc:1905] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:18:57.433879 21111 raft_consensus.cc:2258] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:18:57.443073 21111 ts_tablet_manager.cc:1905] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 01:18:57 dist-test-slave-7nm7 krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:18:57.451889 21112 ts_tablet_manager.cc:1918] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:18:57.453823 21111 ts_tablet_manager.cc:1918] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:18:57.454087 21112 log.cc:1196] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/314467bd7d0f4fa5abbcded0d57b494c
I20260806 01:18:57.454097 21110 ts_tablet_manager.cc:1918] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:18:57.454310 21110 log.cc:1196] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/314467bd7d0f4fa5abbcded0d57b494c
I20260806 01:18:57.454877 21112 ts_tablet_manager.cc:1939] T 314467bd7d0f4fa5abbcded0d57b494c P 7cf1668566d748ed8321d053ba0ff840: Deleting consensus metadata
I20260806 01:18:57.454895 21110 ts_tablet_manager.cc:1939] T 314467bd7d0f4fa5abbcded0d57b494c P f58e9a6107034e1db609d41a12796367: Deleting consensus metadata
I20260806 01:18:57.456326 21111 log.cc:1196] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/314467bd7d0f4fa5abbcded0d57b494c
I20260806 01:18:57.456873 20320 catalog_manager.cc:4841] TS f58e9a6107034e1db609d41a12796367 (127.25.236.130:45369): tablet 314467bd7d0f4fa5abbcded0d57b494c (table test-table [id=b0ee08d400db488fb1b36f7b38a416d4]) successfully deleted
I20260806 01:18:57.457346 21111 ts_tablet_manager.cc:1939] T 314467bd7d0f4fa5abbcded0d57b494c P 53c5ff48ca784062a62cca1be2d1587a: Deleting consensus metadata
I20260806 01:18:57.458174 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.424239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57138 (local address 127.25.236.129:45289)
0806 01:18:57.424480 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:57.424489 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:57.427046 (+ 2557us) server_negotiation.cc:407] Connection header received
0806 01:18:57.427304 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:57.427313 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:57.427485 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:57.427637 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:57.430001 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.431211 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.433194 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.433946 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.434058 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.445785 (+ 11727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:57.445804 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:57.445824 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:57.445880 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:57.450279 (+ 4399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:57.451262 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:57.451271 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:57.451275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:57.451539 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:57.452012 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:57.452019 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:57.452025 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:57.452456 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:57.452748 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:57.457445 (+ 4697us) server_negotiation.cc:299] Negotiation successful
0806 01:18:57.457721 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:18:57.459028 20632 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:18:57.459097 20318 catalog_manager.cc:4841] TS 53c5ff48ca784062a62cca1be2d1587a (127.25.236.129:45289): tablet 314467bd7d0f4fa5abbcded0d57b494c (table test-table [id=b0ee08d400db488fb1b36f7b38a416d4]) successfully deleted
I20260806 01:18:57.459540 20310 catalog_manager.cc:4841] TS 7cf1668566d748ed8321d053ba0ff840 (127.25.236.131:37265): tablet 314467bd7d0f4fa5abbcded0d57b494c (table test-table [id=b0ee08d400db488fb1b36f7b38a416d4]) successfully deleted
I20260806 01:18:57.564323 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 20431
I20260806 01:18:57.591271 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 20647
Aug 06 01:18:57 dist-test-slave-7nm7 krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:18:57.609257 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.592207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60842 (local address 127.25.236.130:45369)
0806 01:18:57.592382 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:57.592387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:57.592406 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:18:57.592467 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:57.592471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:57.592542 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:57.592630 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:57.593842 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.594847 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.596098 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.596485 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.596583 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.603736 (+ 7153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:57.603750 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:57.603766 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:57.603798 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:57.606730 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:57.607429 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:57.607436 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:57.607438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:57.607561 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:57.607902 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:57.607906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:57.607908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:57.608312 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:57.608525 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:57.608826 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 01:18:57.608980 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:18:57.610014 20832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:18:57.700743 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 20647
I20260806 01:18:57.726007 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 20854
Aug 06 01:18:57 dist-test-slave-7nm7 krb5kdc[20271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:18:57.742563 21077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:57.726870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46996 (local address 127.25.236.131:37265)
0806 01:18:57.727058 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:57.727063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:57.727083 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:18:57.727176 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:57.727180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:57.727245 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:57.727329 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:57.728768 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.729505 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.730868 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:57.731266 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:57.731374 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:57.735966 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:57.735983 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:57.735999 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:18:57.736041 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:57.739619 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:57.740383 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:57.740390 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:57.740392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:57.740542 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:18:57.740928 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:18:57.740931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:18:57.740933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:18:57.741385 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:57.741705 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:57.741982 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 01:18:57.742165 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:18:57.743357 21037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:18:57.839506 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 20854
I20260806 01:18:57.863482 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 20287
I20260806 01:18:57.864744 20388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:18:57.983872 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 20287
2026-08-06T01:18:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5101 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T01:18:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:18:58Z Disabled control of system clock
I20260806 01:18:58.094758 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:39787
--webserver_interface=127.25.236.190
--webserver_port=0
--builtin_ntp_servers=127.25.236.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:39787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:18:58.469328 21125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:58.469604 21125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:58.469734 21125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:58.477617 21125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:18:58.477739 21125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:58.477828 21125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:58.477916 21125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:18:58.477978 21125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:18:58.487742 21125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:39787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:39787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:58.489089 21125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:58.490928 21125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:58.503477 21134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:58.503525 21139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:58.503559 21137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:58.504630 21125 server_base.cc:1034] running on GCE node
I20260806 01:18:58.505621 21125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:58.507279 21125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:58.508531 21125 hybrid_clock.cc:648] HybridClock initialized: now 1785979138508470 us; error 52 us; skew 500 ppm
I20260806 01:18:58.508914 21125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:18:58.511674 21125 webserver.cc:460] Webserver started at http://127.25.236.190:36611/ using document root <none> and password file <none>
I20260806 01:18:58.512434 21125 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:58.512574 21125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:58.512924 21125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:18:58.516166 21125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "d0d952aa0acc421f96903cf2ad15d23b"
format_stamp: "Formatted at 2026-08-06 01:18:58 on dist-test-slave-7nm7"
server_key: "7e144ced82cc766ce9183d0369de8ad9"
server_key_iv: "9a1b367dcc51d817ebb728f18458caaa"
server_key_version: "encryptionkey@0"
I20260806 01:18:58.517058 21125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "d0d952aa0acc421f96903cf2ad15d23b"
format_stamp: "Formatted at 2026-08-06 01:18:58 on dist-test-slave-7nm7"
server_key: "7e144ced82cc766ce9183d0369de8ad9"
server_key_iv: "9a1b367dcc51d817ebb728f18458caaa"
server_key_version: "encryptionkey@0"
I20260806 01:18:58.523765 21125 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 01:18:58.527954 21149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:58.530354 21125 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260806 01:18:58.530578 21125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "d0d952aa0acc421f96903cf2ad15d23b"
format_stamp: "Formatted at 2026-08-06 01:18:58 on dist-test-slave-7nm7"
server_key: "7e144ced82cc766ce9183d0369de8ad9"
server_key_iv: "9a1b367dcc51d817ebb728f18458caaa"
server_key_version: "encryptionkey@0"
I20260806 01:18:58.530836 21125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:58.553169 21125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:58.554637 21125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:58.555039 21125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:58.580689 21125 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:39787
I20260806 01:18:58.580701 21230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:39787 every 8 connection(s)
I20260806 01:18:58.582640 21125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:18:58.584482 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 21125
I20260806 01:18:58.584682 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:18:58.585394 26546 external_mini_cluster.cc:1428] Setting key 543e66c7a8e65c46c332172943f4a0f3
I20260806 01:18:58.588624 21231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:18:58.604719 21231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b: Bootstrap starting.
I20260806 01:18:58.605014 21237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:58.587839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36460 (local address 127.25.236.190:39787)
0806 01:18:58.589507 (+ 1668us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:58.589523 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:58.589586 (+ 63us) server_negotiation.cc:407] Connection header received
0806 01:18:58.592353 (+ 2767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:58.592384 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:58.593093 (+ 709us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:58.593601 (+ 508us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:58.594167 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:58.596280 (+ 2113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:58.598531 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:58.599576 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:58.599778 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:58.601588 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:58.601611 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:58.601643 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:18:58.602689 (+ 1046us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:58.602783 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:18:58.602821 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:58.603083 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:58.603307 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 01:18:58.603656 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":797,"thread_start_us":440,"threads_started":1}
I20260806 01:18:58.609266 21231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b: Neither blocks nor log segments found. Creating new log.
I20260806 01:18:58.611019 21231 log.cc:824] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:18:58.614842 21231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b: No bootstrap required, opened a new log
I20260806 01:18:58.622412 21231 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0d952aa0acc421f96903cf2ad15d23b" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39787 } }
I20260806 01:18:58.622855 21231 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:18:58.622990 21231 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0d952aa0acc421f96903cf2ad15d23b, State: Initialized, Role: FOLLOWER
I20260806 01:18:58.623767 21231 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0d952aa0acc421f96903cf2ad15d23b" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39787 } }
I20260806 01:18:58.624032 21231 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:18:58.624193 21231 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:18:58.624418 21231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:18:58.627045 21231 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0d952aa0acc421f96903cf2ad15d23b" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39787 } }
I20260806 01:18:58.627837 21231 leader_election.cc:302] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0d952aa0acc421f96903cf2ad15d23b; no voters:
I20260806 01:18:58.628528 21231 leader_election.cc:288] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:18:58.628705 21241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:18:58.629668 21241 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [term 1 LEADER]: Becoming Leader. State: Replica: d0d952aa0acc421f96903cf2ad15d23b, State: Running, Role: LEADER
I20260806 01:18:58.630198 21241 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0d952aa0acc421f96903cf2ad15d23b" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39787 } }
I20260806 01:18:58.631412 21231 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:18:58.634104 21247 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0d952aa0acc421f96903cf2ad15d23b. Latest consensus state: current_term: 1 leader_uuid: "d0d952aa0acc421f96903cf2ad15d23b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0d952aa0acc421f96903cf2ad15d23b" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39787 } } }
I20260806 01:18:58.634683 21247 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [sys.catalog]: This master's current role is: LEADER
I20260806 01:18:58.634105 21245 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0d952aa0acc421f96903cf2ad15d23b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0d952aa0acc421f96903cf2ad15d23b" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39787 } } }
I20260806 01:18:58.635195 21245 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b [sys.catalog]: This master's current role is: LEADER
I20260806 01:18:58.638880 21253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:18:58.645691 21253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:18:58.654904 21253 catalog_manager.cc:1347] Generated new cluster ID: 7775a6b859f2411c9354b1b6f21061fe
I20260806 01:18:58.655025 21253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:18:58.672864 21253 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:18:58.674404 21253 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:18:58.682905 21253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b: Generated new TSK 0
I20260806 01:18:58.683799 21253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:18:58.689911 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39787
--builtin_ntp_servers=127.25.236.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:18:59.071049 21273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:59.071349 21273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:59.071445 21273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:59.079084 21273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:59.079206 21273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:59.079372 21273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:18:59.088697 21273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:59.090299 21273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:59.092372 21273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:59.105996 21280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:59.106123 21281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:59.106156 21284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:59.107098 21273 server_base.cc:1034] running on GCE node
I20260806 01:18:59.107764 21273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:59.108672 21273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:59.109887 21273 hybrid_clock.cc:648] HybridClock initialized: now 1785979139109839 us; error 46 us; skew 500 ppm
I20260806 01:18:59.110303 21273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:18:59.112792 21273 webserver.cc:460] Webserver started at http://127.25.236.129:42159/ using document root <none> and password file <none>
I20260806 01:18:59.113526 21273 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:59.113626 21273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:59.113883 21273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:18:59.116788 21273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "8d36bfaa798445febb80f66830846d3e"
format_stamp: "Formatted at 2026-08-06 01:18:59 on dist-test-slave-7nm7"
server_key: "00bc105e2bce2eca0812ea094a052f7c"
server_key_iv: "97c3188322863d9cd4082199caa7b4f3"
server_key_version: "encryptionkey@0"
I20260806 01:18:59.117547 21273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "8d36bfaa798445febb80f66830846d3e"
format_stamp: "Formatted at 2026-08-06 01:18:59 on dist-test-slave-7nm7"
server_key: "00bc105e2bce2eca0812ea094a052f7c"
server_key_iv: "97c3188322863d9cd4082199caa7b4f3"
server_key_version: "encryptionkey@0"
I20260806 01:18:59.122860 21273 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:18:59.126598 21292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:59.128011 21273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:18:59.128170 21273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "8d36bfaa798445febb80f66830846d3e"
format_stamp: "Formatted at 2026-08-06 01:18:59 on dist-test-slave-7nm7"
server_key: "00bc105e2bce2eca0812ea094a052f7c"
server_key_iv: "97c3188322863d9cd4082199caa7b4f3"
server_key_version: "encryptionkey@0"
I20260806 01:18:59.128398 21273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:59.144335 21273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:59.145197 21273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:59.145530 21273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:59.146595 21273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:18:59.148085 21273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:18:59.148218 21273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:59.148365 21273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:18:59.148463 21273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:59.183434 21273 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:34601
I20260806 01:18:59.183480 21463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:34601 every 8 connection(s)
I20260806 01:18:59.184749 21273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:18:59.192523 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 21273
I20260806 01:18:59.192715 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:18:59.193275 26546 external_mini_cluster.cc:1428] Setting key 2a963a7401e404e02238c023602f0556
I20260806 01:18:59.196496 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39787
--builtin_ntp_servers=127.25.236.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:18:59.198843 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:59.187413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:37373 (local address 127.25.236.190:39787)
0806 01:18:59.187863 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:59.187868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:59.189065 (+ 1197us) server_negotiation.cc:407] Connection header received
0806 01:18:59.190945 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:59.190952 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:59.191040 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:59.191169 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:59.192521 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:59.193568 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:59.195713 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:59.196132 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:59.196238 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:59.197974 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:59.197979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:59.197983 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:18:59.198002 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:59.198044 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:18:59.198057 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:59.198150 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:59.198483 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 01:18:59.198567 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":224,"threads_started":1}
I20260806 01:18:59.200481 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:59.188000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39787 (local address 127.25.236.129:37373)
0806 01:18:59.188905 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:59.188942 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:59.190667 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:59.191402 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:59.191409 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:59.191492 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:18:59.192295 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:59.192329 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:59.193752 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:59.193758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:59.195558 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:59.195575 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:59.196382 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:59.196390 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:59.196707 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:59.197561 (+ 854us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:18:59.197567 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:18:59.197625 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:18:59.197738 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:18:59.198187 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:18:59.198202 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:18:59.198214 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 01:18:59.198457 (+ 243us) client_negotiation.cc:240] Negotiation successful
0806 01:18:59.198905 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":302,"threads_started":1}
I20260806 01:18:59.202339 21464 heartbeater.cc:347] Connected to a master server at 127.25.236.190:39787
I20260806 01:18:59.202668 21464 heartbeater.cc:464] Registering TS with master...
I20260806 01:18:59.203459 21464 heartbeater.cc:511] Master 127.25.236.190:39787 requested a full tablet report, sending...
I20260806 01:18:59.205516 21177 ts_manager.cc:194] Registered new tserver with Master: 8d36bfaa798445febb80f66830846d3e (127.25.236.129:34601)
I20260806 01:18:59.207051 21177 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.236.129:37373
W20260806 01:18:59.552305 21469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:18:59.552564 21469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:18:59.552665 21469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:18:59.559850 21469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:18:59.559945 21469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:18:59.560046 21469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:18:59.568991 21469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:18:59.570140 21469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:18:59.571846 21469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:18:59.584913 21481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:59.585095 21485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:18:59.585497 21483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:18:59.586320 21469 server_base.cc:1034] running on GCE node
I20260806 01:18:59.586989 21469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:18:59.588001 21469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:18:59.589250 21469 hybrid_clock.cc:648] HybridClock initialized: now 1785979139589192 us; error 43 us; skew 500 ppm
I20260806 01:18:59.589581 21469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:18:59.592092 21469 webserver.cc:460] Webserver started at http://127.25.236.130:41477/ using document root <none> and password file <none>
I20260806 01:18:59.592954 21469 fs_manager.cc:359] Metadata directory not provided
I20260806 01:18:59.593109 21469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:18:59.593431 21469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:18:59.596710 21469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "bb261adfcf144a4a810006e7a7954eeb"
format_stamp: "Formatted at 2026-08-06 01:18:59 on dist-test-slave-7nm7"
server_key: "09b75ab82acbb2f0d78a4ae7b077f154"
server_key_iv: "eeb39c7d054215ca05eb926c0b155cdd"
server_key_version: "encryptionkey@0"
I20260806 01:18:59.597556 21469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "bb261adfcf144a4a810006e7a7954eeb"
format_stamp: "Formatted at 2026-08-06 01:18:59 on dist-test-slave-7nm7"
server_key: "09b75ab82acbb2f0d78a4ae7b077f154"
server_key_iv: "eeb39c7d054215ca05eb926c0b155cdd"
server_key_version: "encryptionkey@0"
I20260806 01:18:59.603325 21469 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 01:18:59.607167 21492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:59.608705 21469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:18:59.608994 21469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "bb261adfcf144a4a810006e7a7954eeb"
format_stamp: "Formatted at 2026-08-06 01:18:59 on dist-test-slave-7nm7"
server_key: "09b75ab82acbb2f0d78a4ae7b077f154"
server_key_iv: "eeb39c7d054215ca05eb926c0b155cdd"
server_key_version: "encryptionkey@0"
I20260806 01:18:59.609239 21469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:18:59.625326 21469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:18:59.626190 21469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:18:59.626575 21469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:18:59.627619 21469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:18:59.629298 21469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:18:59.629446 21469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:59.629594 21469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:18:59.629688 21469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:18:59.671015 21469 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:39005
I20260806 01:18:59.671060 21658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:39005 every 8 connection(s)
I20260806 01:18:59.672358 21469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:18:59.681600 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 21469
I20260806 01:18:59.681977 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:18:59.682614 26546 external_mini_cluster.cc:1428] Setting key 239d709200e198dafda060cd9a5ddb7e
I20260806 01:18:59.685776 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39787
--builtin_ntp_servers=127.25.236.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:18:59.687665 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:59.675613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:43817 (local address 127.25.236.190:39787)
0806 01:18:59.675822 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 01:18:59.675827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:18:59.678323 (+ 2496us) server_negotiation.cc:407] Connection header received
0806 01:18:59.680292 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:18:59.680300 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:18:59.680392 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:18:59.680514 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:18:59.681755 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:59.682661 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:59.684683 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:18:59.685133 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:18:59.685276 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:59.686756 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:18:59.686768 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:18:59.686772 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:18:59.686792 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:18:59.686823 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:18:59.686837 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:18:59.686936 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 01:18:59.687195 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 01:18:59.687298 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 01:18:59.688723 21662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:18:59.676470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39787 (local address 127.25.236.130:43817)
0806 01:18:59.678019 (+ 1549us) negotiation.cc:107] Waiting for socket to connect
0806 01:18:59.678077 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 01:18:59.679915 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:18:59.680771 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:18:59.680780 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:18:59.680877 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:18:59.681484 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:59.681501 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:59.682859 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:59.682866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:59.684530 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:18:59.684546 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:18:59.685375 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:18:59.685382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:18:59.685627 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:18:59.686402 (+ 775us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:18:59.686405 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:18:59.686464 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:18:59.686549 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:18:59.686936 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:18:59.686950 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:18:59.686960 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 01:18:59.687171 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 01:18:59.687600 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"thread_start_us":532,"threads_started":1}
I20260806 01:18:59.690333 21659 heartbeater.cc:347] Connected to a master server at 127.25.236.190:39787
I20260806 01:18:59.690647 21659 heartbeater.cc:464] Registering TS with master...
I20260806 01:18:59.691480 21659 heartbeater.cc:511] Master 127.25.236.190:39787 requested a full tablet report, sending...
I20260806 01:18:59.693106 21177 ts_manager.cc:194] Registered new tserver with Master: bb261adfcf144a4a810006e7a7954eeb (127.25.236.130:39005)
I20260806 01:18:59.694397 21177 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.236.130:43817
W20260806 01:19:00.037171 21663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:00.037454 21663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:00.037544 21663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:00.045007 21663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:00.045120 21663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:00.045228 21663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:19:00.054574 21663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:39787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:00.055817 21663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:00.057587 21663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:00.071115 21672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:00.071161 21671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:00.071197 21674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:00.075946 21663 server_base.cc:1034] running on GCE node
I20260806 01:19:00.076655 21663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:00.077725 21663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:00.079001 21663 hybrid_clock.cc:648] HybridClock initialized: now 1785979140078938 us; error 48 us; skew 500 ppm
I20260806 01:19:00.079701 21663 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:19:00.082533 21663 webserver.cc:460] Webserver started at http://127.25.236.131:38833/ using document root <none> and password file <none>
I20260806 01:19:00.083264 21663 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:00.083403 21663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:00.083725 21663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:00.086795 21663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "0c789d6498734bccb32b958843b12fda"
format_stamp: "Formatted at 2026-08-06 01:19:00 on dist-test-slave-7nm7"
server_key: "3a19adf132d524e648d0f252367a668f"
server_key_iv: "9d579570220e91048217f6f948372e2b"
server_key_version: "encryptionkey@0"
I20260806 01:19:00.087625 21663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "0c789d6498734bccb32b958843b12fda"
format_stamp: "Formatted at 2026-08-06 01:19:00 on dist-test-slave-7nm7"
server_key: "3a19adf132d524e648d0f252367a668f"
server_key_iv: "9d579570220e91048217f6f948372e2b"
server_key_version: "encryptionkey@0"
I20260806 01:19:00.093516 21663 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:19:00.098486 21685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:00.100356 21663 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:19:00.100594 21663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "0c789d6498734bccb32b958843b12fda"
format_stamp: "Formatted at 2026-08-06 01:19:00 on dist-test-slave-7nm7"
server_key: "3a19adf132d524e648d0f252367a668f"
server_key_iv: "9d579570220e91048217f6f948372e2b"
server_key_version: "encryptionkey@0"
I20260806 01:19:00.100854 21663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:00.130182 21663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:00.131157 21663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:00.131479 21663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:00.132671 21663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:19:00.134765 21663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:19:00.134897 21663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:00.135036 21663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:19:00.135138 21663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:00.175086 21663 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:40007
I20260806 01:19:00.175099 21843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:40007 every 8 connection(s)
I20260806 01:19:00.176560 21663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:19:00.185371 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 21663
I20260806 01:19:00.185617 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:19:00.186272 26546 external_mini_cluster.cc:1428] Setting key 103387db18ff0ecc62fad8781c504ca5
I20260806 01:19:00.191640 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.179597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:58325 (local address 127.25.236.190:39787)
0806 01:19:00.179822 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:00.179832 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:00.181238 (+ 1406us) server_negotiation.cc:407] Connection header received
0806 01:19:00.183783 (+ 2545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:00.183788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:00.183861 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:00.183943 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:00.185515 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.186684 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.188810 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.189345 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.189472 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.190902 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:00.190907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:00.190910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:00.190937 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:00.190969 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:00.190982 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:00.191060 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:00.191316 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 01:19:00.191394 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:19:00.192575 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.180111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:39787 (local address 127.25.236.131:58325)
0806 01:19:00.181044 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:00.181099 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:00.183446 (+ 2347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:00.184240 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:00.184251 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:00.184379 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:19:00.185268 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:00.185298 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.186856 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.186863 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:00.188666 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:00.188679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.189593 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.189611 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:00.189840 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.190593 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:19:00.190596 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:19:00.190638 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:19:00.190709 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:19:00.191083 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:19:00.191094 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:19:00.191109 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 01:19:00.191268 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 01:19:00.191548 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":319,"threads_started":1}
I20260806 01:19:00.194381 21844 heartbeater.cc:347] Connected to a master server at 127.25.236.190:39787
I20260806 01:19:00.194659 21844 heartbeater.cc:464] Registering TS with master...
I20260806 01:19:00.195375 21844 heartbeater.cc:511] Master 127.25.236.190:39787 requested a full tablet report, sending...
I20260806 01:19:00.196777 21177 ts_manager.cc:194] Registered new tserver with Master: 0c789d6498734bccb32b958843b12fda (127.25.236.131:40007)
I20260806 01:19:00.197642 21177 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.236.131:58325
I20260806 01:19:00.202039 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:19:00.209862 21464 heartbeater.cc:503] Master 127.25.236.190:39787 was elected leader, sending a full tablet report...
I20260806 01:19:00.213904 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.206874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36464 (local address 127.25.236.190:39787)
0806 01:19:00.207177 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:00.207187 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:00.207608 (+ 421us) server_negotiation.cc:407] Connection header received
0806 01:19:00.207741 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:00.207746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:00.207821 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:00.208073 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:00.208495 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.209229 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.210998 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.211649 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.211795 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.212957 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:00.212962 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:00.212965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:00.212983 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:00.213014 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:00.213026 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:00.213265 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:00.213422 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 01:19:00.213517 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:19:00.218195 21177 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:19:00.225754 21177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:19:00.249974 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.237953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34601 (local address 127.0.0.1:60358)
0806 01:19:00.239063 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:00.239100 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:00.239366 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:00.242689 (+ 3323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:00.242693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:00.242728 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:00.243077 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:00.243109 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.245085 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.245092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:00.247034 (+ 1942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:00.247047 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.248894 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.248901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:00.249152 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.249214 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 01:19:00.249314 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 01:19:00.249497 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"mutex_wait_us":105,"thread_start_us":692,"threads_started":1}
I20260806 01:19:00.249998 21860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.238066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:40007 (local address 127.0.0.1:35864)
0806 01:19:00.241218 (+ 3152us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:00.241241 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:00.241390 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:00.242206 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:00.242215 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:00.242280 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:00.242765 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:00.242779 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.244633 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.244640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:00.247359 (+ 2719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:00.247376 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.249169 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.249176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:00.249341 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.249368 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 01:19:00.249424 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 01:19:00.249520 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2967,"mutex_wait_us":143,"thread_start_us":2780,"threads_started":1}
I20260806 01:19:00.250531 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.239704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:39005 (local address 127.0.0.1:36464)
0806 01:19:00.242309 (+ 2605us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:00.242327 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:00.242464 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:00.243173 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:00.243179 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:00.243209 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:00.243606 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:00.243618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.245405 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.245413 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:00.247712 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:00.247722 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.249401 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.249408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:00.249588 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.249619 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 01:19:00.249717 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 01:19:00.249846 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2460,"mutex_wait_us":51,"spinlock_wait_cycles":5376,"thread_start_us":2317,"threads_started":1}
I20260806 01:19:00.250587 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.240179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36464 (local address 127.25.236.130:39005)
0806 01:19:00.241887 (+ 1708us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:00.241893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:00.242594 (+ 701us) server_negotiation.cc:407] Connection header received
0806 01:19:00.242699 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:00.242708 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:00.242927 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:00.243158 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:00.243776 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.245229 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.247880 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.249195 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.249386 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.249621 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:00.249758 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 01:19:00.249918 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1559,"thread_start_us":1458,"threads_started":1}
I20260806 01:19:00.251039 21864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.238879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35864 (local address 127.25.236.131:40007)
0806 01:19:00.240357 (+ 1478us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:00.240370 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:00.241547 (+ 1177us) server_negotiation.cc:407] Connection header received
0806 01:19:00.241657 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:00.241668 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:00.241961 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:00.242190 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:00.242989 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.244457 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.247563 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.248953 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.250257 (+ 1304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.250447 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:00.250515 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 01:19:00.250691 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1257,"thread_start_us":1078,"threads_started":1}
I20260806 01:19:00.251431 21861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.238351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.25.236.129:34601)
0806 01:19:00.240930 (+ 2579us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:00.240942 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:00.240978 (+ 36us) server_negotiation.cc:407] Connection header received
0806 01:19:00.241094 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:00.241105 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:00.241413 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:00.242693 (+ 1280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:00.243315 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.244939 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.247234 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.248668 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.250702 (+ 2034us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.250893 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:00.250954 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 01:19:00.251128 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2363,"thread_start_us":2145,"threads_started":1}
I20260806 01:19:00.254432 21754 tablet_service.cc:1467] Processing CreateTablet for tablet 6719371990cb4389ba4f7e11686102e2 (DEFAULT_TABLE table=test-table [id=2daa35d90d184379a9b2898cf3298c8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:00.254434 21560 tablet_service.cc:1467] Processing CreateTablet for tablet 6719371990cb4389ba4f7e11686102e2 (DEFAULT_TABLE table=test-table [id=2daa35d90d184379a9b2898cf3298c8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:00.255596 21560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6719371990cb4389ba4f7e11686102e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:00.255625 21754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6719371990cb4389ba4f7e11686102e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:00.260120 21360 tablet_service.cc:1467] Processing CreateTablet for tablet 6719371990cb4389ba4f7e11686102e2 (DEFAULT_TABLE table=test-table [id=2daa35d90d184379a9b2898cf3298c8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:00.261539 21360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6719371990cb4389ba4f7e11686102e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:00.267077 21868 tablet_bootstrap.cc:492] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb: Bootstrap starting.
I20260806 01:19:00.269480 21869 tablet_bootstrap.cc:492] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda: Bootstrap starting.
I20260806 01:19:00.270562 21868 tablet_bootstrap.cc:654] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:00.271935 21868 log.cc:824] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:00.272940 21869 tablet_bootstrap.cc:654] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:00.274058 21869 log.cc:824] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:00.275243 21868 tablet_bootstrap.cc:492] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb: No bootstrap required, opened a new log
I20260806 01:19:00.275627 21868 ts_tablet_manager.cc:1397] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260806 01:19:00.277495 21869 tablet_bootstrap.cc:492] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda: No bootstrap required, opened a new log
I20260806 01:19:00.277758 21869 ts_tablet_manager.cc:1397] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 01:19:00.286423 21868 raft_consensus.cc:348] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d36bfaa798445febb80f66830846d3e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34601 } } peers { permanent_uuid: "0c789d6498734bccb32b958843b12fda" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 40007 } } peers { permanent_uuid: "bb261adfcf144a4a810006e7a7954eeb" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 39005 } }
I20260806 01:19:00.286938 21868 raft_consensus.cc:374] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:00.287043 21868 raft_consensus.cc:729] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb261adfcf144a4a810006e7a7954eeb, State: Initialized, Role: FOLLOWER
I20260806 01:19:00.287357 21876 tablet_bootstrap.cc:492] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e: Bootstrap starting.
I20260806 01:19:00.287804 21868 consensus_queue.cc:260] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d36bfaa798445febb80f66830846d3e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34601 } } peers { permanent_uuid: "0c789d6498734bccb32b958843b12fda" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 40007 } } peers { permanent_uuid: "bb261adfcf144a4a810006e7a7954eeb" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 39005 } }
I20260806 01:19:00.289026 21659 heartbeater.cc:503] Master 127.25.236.190:39787 was elected leader, sending a full tablet report...
I20260806 01:19:00.289649 21868 ts_tablet_manager.cc:1428] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260806 01:19:00.291721 21876 tablet_bootstrap.cc:654] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:00.291610 21869 raft_consensus.cc:348] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d36bfaa798445febb80f66830846d3e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34601 } } peers { permanent_uuid: "0c789d6498734bccb32b958843b12fda" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 40007 } } peers { permanent_uuid: "bb261adfcf144a4a810006e7a7954eeb" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 39005 } }
I20260806 01:19:00.292039 21869 raft_consensus.cc:374] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:00.292155 21869 raft_consensus.cc:729] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c789d6498734bccb32b958843b12fda, State: Initialized, Role: FOLLOWER
I20260806 01:19:00.292773 21869 consensus_queue.cc:260] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d36bfaa798445febb80f66830846d3e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34601 } } peers { permanent_uuid: "0c789d6498734bccb32b958843b12fda" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 40007 } } peers { permanent_uuid: "bb261adfcf144a4a810006e7a7954eeb" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 39005 } }
I20260806 01:19:00.293076 21876 log.cc:824] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:00.294250 21844 heartbeater.cc:503] Master 127.25.236.190:39787 was elected leader, sending a full tablet report...
I20260806 01:19:00.294906 21869 ts_tablet_manager.cc:1428] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda: Time spent starting tablet: real 0.017s user 0.011s sys 0.005s
I20260806 01:19:00.298170 21876 tablet_bootstrap.cc:492] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e: No bootstrap required, opened a new log
I20260806 01:19:00.298506 21876 ts_tablet_manager.cc:1397] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260806 01:19:00.306622 21876 raft_consensus.cc:348] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d36bfaa798445febb80f66830846d3e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34601 } } peers { permanent_uuid: "0c789d6498734bccb32b958843b12fda" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 40007 } } peers { permanent_uuid: "bb261adfcf144a4a810006e7a7954eeb" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 39005 } }
I20260806 01:19:00.306968 21876 raft_consensus.cc:374] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:00.307083 21876 raft_consensus.cc:729] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d36bfaa798445febb80f66830846d3e, State: Initialized, Role: FOLLOWER
I20260806 01:19:00.307718 21876 consensus_queue.cc:260] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d36bfaa798445febb80f66830846d3e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34601 } } peers { permanent_uuid: "0c789d6498734bccb32b958843b12fda" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 40007 } } peers { permanent_uuid: "bb261adfcf144a4a810006e7a7954eeb" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 39005 } }
I20260806 01:19:00.309159 21876 ts_tablet_manager.cc:1428] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e: Time spent starting tablet: real 0.011s user 0.002s sys 0.008s
W20260806 01:19:00.424784 21660 tablet.cc:2369] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:19:00.428395 21845 tablet.cc:2369] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:19:00.437355 21466 tablet.cc:2369] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:19:00.581629 21882 raft_consensus.cc:482] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:19:00.581964 21882 raft_consensus.cc:504] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d36bfaa798445febb80f66830846d3e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34601 } } peers { permanent_uuid: "0c789d6498734bccb32b958843b12fda" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 40007 } } peers { permanent_uuid: "bb261adfcf144a4a810006e7a7954eeb" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 39005 } }
I20260806 01:19:00.584053 21882 leader_election.cc:288] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c789d6498734bccb32b958843b12fda (127.25.236.131:40007), bb261adfcf144a4a810006e7a7954eeb (127.25.236.130:39005)
I20260806 01:19:00.593531 21885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.584448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:40007 (local address 127.25.236.129:53565)
0806 01:19:00.586721 (+ 2273us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:00.586740 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:00.586902 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:00.587254 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:00.587259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:00.587359 (+ 100us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:00.587752 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:00.587763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.589107 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.589114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:00.590699 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:00.590712 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.592094 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.592100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:00.592382 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.592607 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 01:19:00.592985 (+ 378us) client_negotiation.cc:240] Negotiation successful
0806 01:19:00.593142 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2150,"mutex_wait_us":43,"spinlock_wait_cycles":22912,"thread_start_us":1522,"threads_started":1}
I20260806 01:19:00.593621 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.585263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:43733 (local address 127.25.236.130:39005)
0806 01:19:00.585755 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:00.585764 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:00.586510 (+ 746us) server_negotiation.cc:407] Connection header received
0806 01:19:00.586844 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:00.586851 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:00.586943 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:00.587048 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:00.587695 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.588785 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.591485 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.592512 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.592710 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.592776 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:00.593197 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 01:19:00.593356 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314}
I20260806 01:19:00.593622 21864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.584824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:53565 (local address 127.25.236.131:40007)
0806 01:19:00.585539 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:00.585549 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:00.586813 (+ 1264us) server_negotiation.cc:407] Connection header received
0806 01:19:00.587048 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:00.587053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:00.587132 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:00.587231 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:00.587942 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.588970 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.590898 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.591897 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.592060 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.592151 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:00.593042 (+ 891us) server_negotiation.cc:299] Negotiation successful
0806 01:19:00.593297 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526}
I20260806 01:19:00.594687 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.584363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:39005 (local address 127.25.236.129:43733)
0806 01:19:00.586364 (+ 2001us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:00.586396 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:00.586660 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:00.587064 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:00.587068 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:00.587092 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:00.587538 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:00.587549 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.588931 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.588936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:00.591285 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:00.591300 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.592735 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.592743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:00.592896 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.592920 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:19:00.593805 (+ 885us) client_negotiation.cc:240] Negotiation successful
0806 01:19:00.593955 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1810,"mutex_wait_us":59,"thread_start_us":2039,"threads_started":1}
I20260806 01:19:00.595907 21783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6719371990cb4389ba4f7e11686102e2" candidate_uuid: "8d36bfaa798445febb80f66830846d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c789d6498734bccb32b958843b12fda" is_pre_election: true
I20260806 01:19:00.596374 21592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6719371990cb4389ba4f7e11686102e2" candidate_uuid: "8d36bfaa798445febb80f66830846d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb261adfcf144a4a810006e7a7954eeb" is_pre_election: true
I20260806 01:19:00.596942 21783 raft_consensus.cc:2454] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d36bfaa798445febb80f66830846d3e in term 0.
I20260806 01:19:00.597345 21592 raft_consensus.cc:2454] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d36bfaa798445febb80f66830846d3e in term 0.
I20260806 01:19:00.598270 21293 leader_election.cc:302] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c789d6498734bccb32b958843b12fda, 8d36bfaa798445febb80f66830846d3e; no voters:
I20260806 01:19:00.598784 21882 raft_consensus.cc:2781] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:19:00.598963 21882 raft_consensus.cc:482] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:19:00.599215 21882 raft_consensus.cc:3037] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:00.601751 21882 raft_consensus.cc:504] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d36bfaa798445febb80f66830846d3e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34601 } } peers { permanent_uuid: "0c789d6498734bccb32b958843b12fda" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 40007 } } peers { permanent_uuid: "bb261adfcf144a4a810006e7a7954eeb" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 39005 } }
I20260806 01:19:00.602521 21882 leader_election.cc:288] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [CANDIDATE]: Term 1 election: Requested vote from peers 0c789d6498734bccb32b958843b12fda (127.25.236.131:40007), bb261adfcf144a4a810006e7a7954eeb (127.25.236.130:39005)
I20260806 01:19:00.603034 21783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6719371990cb4389ba4f7e11686102e2" candidate_uuid: "8d36bfaa798445febb80f66830846d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c789d6498734bccb32b958843b12fda"
I20260806 01:19:00.603060 21592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6719371990cb4389ba4f7e11686102e2" candidate_uuid: "8d36bfaa798445febb80f66830846d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb261adfcf144a4a810006e7a7954eeb"
I20260806 01:19:00.603286 21783 raft_consensus.cc:3037] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:00.603332 21592 raft_consensus.cc:3037] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:00.607182 21592 raft_consensus.cc:2454] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d36bfaa798445febb80f66830846d3e in term 1.
I20260806 01:19:00.607221 21783 raft_consensus.cc:2454] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d36bfaa798445febb80f66830846d3e in term 1.
I20260806 01:19:00.607761 21296 leader_election.cc:302] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d36bfaa798445febb80f66830846d3e, bb261adfcf144a4a810006e7a7954eeb; no voters:
I20260806 01:19:00.608095 21882 raft_consensus.cc:2781] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:00.608631 21882 raft_consensus.cc:686] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [term 1 LEADER]: Becoming Leader. State: Replica: 8d36bfaa798445febb80f66830846d3e, State: Running, Role: LEADER
I20260806 01:19:00.609264 21882 consensus_queue.cc:237] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d36bfaa798445febb80f66830846d3e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34601 } } peers { permanent_uuid: "0c789d6498734bccb32b958843b12fda" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 40007 } } peers { permanent_uuid: "bb261adfcf144a4a810006e7a7954eeb" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 39005 } }
I20260806 01:19:00.615463 21175 catalog_manager.cc:5495] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d36bfaa798445febb80f66830846d3e (127.25.236.129). New cstate: current_term: 1 leader_uuid: "8d36bfaa798445febb80f66830846d3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d36bfaa798445febb80f66830846d3e" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 34601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c789d6498734bccb32b958843b12fda" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 40007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb261adfcf144a4a810006e7a7954eeb" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 39005 } health_report { overall_health: UNKNOWN } } }
I20260806 01:19:00.710160 21861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.704820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60368 (local address 127.25.236.129:34601)
0806 01:19:00.705038 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:00.705047 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:00.705086 (+ 39us) server_negotiation.cc:407] Connection header received
0806 01:19:00.705189 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:00.705193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:00.705287 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:00.705411 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:19:00.705789 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.706548 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.707962 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.708436 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.708540 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.709023 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:19:00.709554 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:19:00.709663 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:00.709785 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 01:19:00.709850 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:19:00.718966 21592 raft_consensus.cc:1261] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb [term 1 FOLLOWER]: Refusing update from remote peer 8d36bfaa798445febb80f66830846d3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:19:00.718966 21783 raft_consensus.cc:1261] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda [term 1 FOLLOWER]: Refusing update from remote peer 8d36bfaa798445febb80f66830846d3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:19:00.719982 21887 consensus_queue.cc:1035] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c789d6498734bccb32b958843b12fda" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 40007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:19:00.720328 21882 consensus_queue.cc:1035] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb261adfcf144a4a810006e7a7954eeb" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 39005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:19:00.732766 21893 mvcc.cc:205] Tried to move back new op lower bound from 7315370560375533568 to 7315370559941009408. Current Snapshot: MvccSnapshot[applied={T|T < 7315370560375533568}]
I20260806 01:19:00.747922 21177 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:19:00.748212 21177 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:19:00.751852 21177 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0d952aa0acc421f96903cf2ad15d23b: Sending DeleteTablet for 3 replicas of tablet 6719371990cb4389ba4f7e11686102e2
I20260806 01:19:00.753002 21360 tablet_service.cc:1514] Processing DeleteTablet for tablet 6719371990cb4389ba4f7e11686102e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:00 UTC) from {username='slave'} at 127.0.0.1:60358
I20260806 01:19:00.753106 21754 tablet_service.cc:1514] Processing DeleteTablet for tablet 6719371990cb4389ba4f7e11686102e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:00 UTC) from {username='slave'} at 127.0.0.1:35864
I20260806 01:19:00.753454 21560 tablet_service.cc:1514] Processing DeleteTablet for tablet 6719371990cb4389ba4f7e11686102e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:00 UTC) from {username='slave'} at 127.0.0.1:36464
I20260806 01:19:00.755771 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 21273
I20260806 01:19:00.764200 21912 tablet_replica.cc:330] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb: stopping tablet replica
I20260806 01:19:00.764353 21907 tablet_replica.cc:330] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e: stopping tablet replica
I20260806 01:19:00.764752 21912 raft_consensus.cc:2229] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:19:00.764771 21907 raft_consensus.cc:2229] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:19:00.765323 21912 raft_consensus.cc:2258] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:00.765551 21907 raft_consensus.cc:2258] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:00.766369 21908 tablet_replica.cc:330] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda: stopping tablet replica
I20260806 01:19:00.766789 21908 raft_consensus.cc:2229] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:19:00.767294 21908 raft_consensus.cc:2258] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:00.768421 21907 ts_tablet_manager.cc:1905] T 6719371990cb4389ba4f7e11686102e2 P 8d36bfaa798445febb80f66830846d3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:00.769080 21908 ts_tablet_manager.cc:1905] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:00.770606 21912 ts_tablet_manager.cc:1905] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:00.776374 21908 ts_tablet_manager.cc:1918] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:00.776571 21908 log.cc:1196] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/6719371990cb4389ba4f7e11686102e2
I20260806 01:19:00.777072 21908 ts_tablet_manager.cc:1939] T 6719371990cb4389ba4f7e11686102e2 P 0c789d6498734bccb32b958843b12fda: Deleting consensus metadata
I20260806 01:19:00.777881 21912 ts_tablet_manager.cc:1918] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:00.778051 21912 log.cc:1196] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/6719371990cb4389ba4f7e11686102e2
I20260806 01:19:00.778808 21912 ts_tablet_manager.cc:1939] T 6719371990cb4389ba4f7e11686102e2 P bb261adfcf144a4a810006e7a7954eeb: Deleting consensus metadata
I20260806 01:19:00.779546 21150 catalog_manager.cc:4841] TS 0c789d6498734bccb32b958843b12fda (127.25.236.131:40007): tablet 6719371990cb4389ba4f7e11686102e2 (table test-table [id=2daa35d90d184379a9b2898cf3298c8a]) successfully deleted
I20260806 01:19:00.780514 21151 catalog_manager.cc:4841] TS bb261adfcf144a4a810006e7a7954eeb (127.25.236.130:39005): tablet 6719371990cb4389ba4f7e11686102e2 (table test-table [id=2daa35d90d184379a9b2898cf3298c8a]) successfully deleted
I20260806 01:19:00.780777 21861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.757691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60374 (local address 127.25.236.129:34601)
0806 01:19:00.757940 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:00.757950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:00.769807 (+ 11857us) server_negotiation.cc:407] Connection header received
0806 01:19:00.770356 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:00.770362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:00.770462 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:00.770548 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:00.771365 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.772749 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.774889 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.776006 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.776303 (+ 297us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.778789 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:00.778800 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:00.778819 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:00.779683 (+ 864us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:00.779752 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:00.779785 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:00.779879 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:00.780257 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 01:19:00.780383 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:19:00.781958 21457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:00.886547 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 21273
I20260806 01:19:00.913257 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 21469
W20260806 01:19:00.913271 21151 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.236.129:34601 (error 108)
I20260806 01:19:00.915400 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.914610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34601 (local address 127.0.0.1:60390)
0806 01:19:00.915085 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:00.915204 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":258,"threads_started":1}
W20260806 01:19:00.915681 21151 catalog_manager.cc:4584] TS 8d36bfaa798445febb80f66830846d3e (127.25.236.129:34601): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6719371990cb4389ba4f7e11686102e2: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34601: connect: Connection refused (error 111)
I20260806 01:19:00.921815 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.914170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36476 (local address 127.25.236.130:39005)
0806 01:19:00.914405 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:00.914412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:00.914497 (+ 85us) server_negotiation.cc:407] Connection header received
0806 01:19:00.914714 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:00.914719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:00.914811 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:00.914933 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:00.915358 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.916246 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.918107 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:00.918792 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:00.918948 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:00.920101 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:00.920109 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:00.920124 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:00.920748 (+ 624us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:00.920799 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:00.920824 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:00.920968 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:00.921262 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 01:19:00.921467 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 01:19:00.922569 21652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:00.969889 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.969410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34601 (local address 127.0.0.1:60392)
0806 01:19:00.969653 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:00.969740 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 01:19:00.971004 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:00.970622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34601 (local address 127.0.0.1:60396)
0806 01:19:00.970796 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:00.970870 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 01:19:01.016927 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 21469
I20260806 01:19:01.031762 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:01.031220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34601 (local address 127.0.0.1:60402)
0806 01:19:01.031508 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:01.031604 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 01:19:01.033000 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:01.032594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34601 (local address 127.0.0.1:60412)
0806 01:19:01.032792 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:01.032867 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 01:19:01.044711 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 21663
I20260806 01:19:01.053643 21864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:01.045752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35880 (local address 127.25.236.131:40007)
0806 01:19:01.046319 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:01.046325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:01.046345 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:19:01.046425 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:01.046431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:01.046534 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:01.046649 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:01.047143 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:01.047870 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:01.049307 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:01.049710 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:01.049809 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:01.051196 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:01.051205 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:01.051220 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:01.051836 (+ 616us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:01.051884 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:01.051905 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:01.052519 (+ 614us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:01.052621 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 01:19:01.053129 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261}
I20260806 01:19:01.054553 21839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:01.113905 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:01.113425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34601 (local address 127.0.0.1:60428)
0806 01:19:01.113650 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:01.113748 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 01:19:01.115054 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:01.114643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:34601 (local address 127.0.0.1:60432)
0806 01:19:01.114853 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:01.114926 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 01:19:01.150816 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 21663
I20260806 01:19:01.176251 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 21125
I20260806 01:19:01.177467 21226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:01.297454 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 21125
2026-08-06T01:19:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3308 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:01 dist-test-slave-7nm7 krb5kdc[21927](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:01 dist-test-slave-7nm7 krb5kdc[21927](info): set up 2 sockets
Aug 06 01:19:01 dist-test-slave-7nm7 krb5kdc[21927](info): commencing operation
krb5kdc: starting...
W20260806 01:19:02.677870 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:02 dist-test-slave-7nm7 krb5kdc[21927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:19:03.021347 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:40155
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:40155
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:03.383687 21946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:03.383988 21946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:03.384334 21946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:03.391809 21946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:03.391913 21946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:03.392028 21946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:03.392218 21946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:03.392460 21946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:03.403716 21946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:40155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:40155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:03.405180 21946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:03.406483 21946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:03.419052 21954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:03.418987 21955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:03.419095 21958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:03.420101 21946 server_base.cc:1034] running on GCE node
I20260806 01:19:03.420804 21946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:03.422132 21946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:03.423413 21946 hybrid_clock.cc:648] HybridClock initialized: now 1785979143423379 us; error 63 us; skew 500 ppm
I20260806 01:19:03.423856 21946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:03 dist-test-slave-7nm7 krb5kdc[21927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979143, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:03.442713 21946 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:19:03.444468 21946 webserver.cc:460] Webserver started at http://127.25.236.190:38497/ using document root <none> and password file <none>
I20260806 01:19:03.445225 21946 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:03.445380 21946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:03.445709 21946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:03.448904 21946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "ae9f30c69e7b446b91b8d7c14747f603"
format_stamp: "Formatted at 2026-08-06 01:19:03 on dist-test-slave-7nm7"
server_key: "2c9ef74854a0b08067c132a8a5df5896"
server_key_iv: "8bb411e8c198f0f23fe2d44a33d19ce8"
server_key_version: "encryptionkey@0"
I20260806 01:19:03.449689 21946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "ae9f30c69e7b446b91b8d7c14747f603"
format_stamp: "Formatted at 2026-08-06 01:19:03 on dist-test-slave-7nm7"
server_key: "2c9ef74854a0b08067c132a8a5df5896"
server_key_iv: "8bb411e8c198f0f23fe2d44a33d19ce8"
server_key_version: "encryptionkey@0"
I20260806 01:19:03.455681 21946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 01:19:03.459192 21968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:03.460500 21946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:19:03.460732 21946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "ae9f30c69e7b446b91b8d7c14747f603"
format_stamp: "Formatted at 2026-08-06 01:19:03 on dist-test-slave-7nm7"
server_key: "2c9ef74854a0b08067c132a8a5df5896"
server_key_iv: "8bb411e8c198f0f23fe2d44a33d19ce8"
server_key_version: "encryptionkey@0"
I20260806 01:19:03.460937 21946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:03.475597 21946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:03.480967 21946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:03.481397 21946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:03.504235 21946 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:40155
I20260806 01:19:03.504269 22043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:40155 every 8 connection(s)
I20260806 01:19:03.505673 21946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:03.507354 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 21946
I20260806 01:19:03.507560 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:03.508153 26546 external_mini_cluster.cc:1428] Setting key 06b4dd627e8a9aaa4deb18828ff572bc
I20260806 01:19:03.510514 22045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:19:03 dist-test-slave-7nm7 krb5kdc[21927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:19:03.527050 22045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603: Bootstrap starting.
I20260806 01:19:03.532449 22045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:03.533849 22045 log.cc:824] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:03.536978 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:03.510155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40176 (local address 127.25.236.190:40155)
0806 01:19:03.511197 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:03.511210 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:03.511253 (+ 43us) server_negotiation.cc:407] Connection header received
0806 01:19:03.513274 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:03.513297 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:03.513668 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:03.513999 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:03.516238 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:03.517670 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:03.519612 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:03.520464 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:03.520602 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:03.526051 (+ 5449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:03.526064 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:03.526205 (+ 141us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:03.526452 (+ 247us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:03.531622 (+ 5170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:03.532576 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:03.532586 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:03.532594 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:03.532807 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:03.533347 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:03.533352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:03.533355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:03.534098 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:03.534480 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:03.535254 (+ 774us) server_negotiation.cc:299] Negotiation successful
0806 01:19:03.535670 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":357,"threads_started":1}
I20260806 01:19:03.537626 22045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603: No bootstrap required, opened a new log
I20260806 01:19:03.545600 22045 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9f30c69e7b446b91b8d7c14747f603" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40155 } }
I20260806 01:19:03.545926 22045 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:03.545997 22045 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae9f30c69e7b446b91b8d7c14747f603, State: Initialized, Role: FOLLOWER
I20260806 01:19:03.546538 22045 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9f30c69e7b446b91b8d7c14747f603" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40155 } }
I20260806 01:19:03.546754 22045 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:03.546854 22045 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:03.546983 22045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:03.549075 22045 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9f30c69e7b446b91b8d7c14747f603" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40155 } }
I20260806 01:19:03.549515 22045 leader_election.cc:302] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae9f30c69e7b446b91b8d7c14747f603; no voters:
I20260806 01:19:03.550184 22045 leader_election.cc:288] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:03.550343 22053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:03.551035 22053 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [term 1 LEADER]: Becoming Leader. State: Replica: ae9f30c69e7b446b91b8d7c14747f603, State: Running, Role: LEADER
I20260806 01:19:03.551643 22053 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9f30c69e7b446b91b8d7c14747f603" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40155 } }
I20260806 01:19:03.552657 22045 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:03.554621 22055 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae9f30c69e7b446b91b8d7c14747f603" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9f30c69e7b446b91b8d7c14747f603" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40155 } } }
I20260806 01:19:03.554656 22054 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae9f30c69e7b446b91b8d7c14747f603. Latest consensus state: current_term: 1 leader_uuid: "ae9f30c69e7b446b91b8d7c14747f603" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9f30c69e7b446b91b8d7c14747f603" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 40155 } } }
I20260806 01:19:03.554991 22055 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:03.555068 22054 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:03.561587 22060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:03.567042 22060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:03.576509 22060 catalog_manager.cc:1347] Generated new cluster ID: 15597c2c9b974920a619415606617f73
I20260806 01:19:03.576642 22060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:03.607656 22060 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:03.609634 22060 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:03.620030 22060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603: Generated new TSK 0
I20260806 01:19:03.621096 22060 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:19:03.696012 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:40155
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:04.074071 22082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:04.074473 22082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:04.074680 22082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:04.082836 22082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:04.082964 22082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:04.083169 22082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:19:04.094419 22082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:40155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:04.095913 22082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:04.097257 22082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:04.111580 22092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:04.111670 22094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:04.111670 22096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:04.112365 22082 server_base.cc:1034] running on GCE node
I20260806 01:19:04.113013 22082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:04.113958 22082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:04.115157 22082 hybrid_clock.cc:648] HybridClock initialized: now 1785979144115106 us; error 42 us; skew 500 ppm
I20260806 01:19:04.115481 22082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:04 dist-test-slave-7nm7 krb5kdc[21927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979144, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:04.118687 22082 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:19:04.120430 22082 webserver.cc:460] Webserver started at http://127.25.236.129:35427/ using document root <none> and password file <none>
I20260806 01:19:04.121119 22082 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:04.121253 22082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:04.121585 22082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:04.124506 22082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "e1574685c3434c3386642e828394633b"
format_stamp: "Formatted at 2026-08-06 01:19:04 on dist-test-slave-7nm7"
server_key: "41802f1fed56f04a00a645435e8aac2c"
server_key_iv: "9630bac2b7196493400c62e0f73195f2"
server_key_version: "encryptionkey@0"
I20260806 01:19:04.125290 22082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "e1574685c3434c3386642e828394633b"
format_stamp: "Formatted at 2026-08-06 01:19:04 on dist-test-slave-7nm7"
server_key: "41802f1fed56f04a00a645435e8aac2c"
server_key_iv: "9630bac2b7196493400c62e0f73195f2"
server_key_version: "encryptionkey@0"
I20260806 01:19:04.130414 22082 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:19:04.133795 22105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:04.135214 22082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:04.135382 22082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "e1574685c3434c3386642e828394633b"
format_stamp: "Formatted at 2026-08-06 01:19:04 on dist-test-slave-7nm7"
server_key: "41802f1fed56f04a00a645435e8aac2c"
server_key_iv: "9630bac2b7196493400c62e0f73195f2"
server_key_version: "encryptionkey@0"
I20260806 01:19:04.135592 22082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:04.147584 22082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:04.151556 22082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:04.151852 22082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:04.152793 22082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:19:04.154419 22082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:19:04.154495 22082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:04.154634 22082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:19:04.154699 22082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:04.189306 22082 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:45591
I20260806 01:19:04.189355 22264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:45591 every 8 connection(s)
I20260806 01:19:04.190610 22082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:19:04.191749 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 22082
I20260806 01:19:04.191946 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:19:04.192529 26546 external_mini_cluster.cc:1428] Setting key 6baa0535c77cda602a8c6f6974a08606
Aug 06 01:19:04 dist-test-slave-7nm7 krb5kdc[21927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979144, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:19:04.213738 22272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:04.193924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:59449 (local address 127.25.236.190:40155)
0806 01:19:04.194451 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:04.194455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:04.195843 (+ 1388us) server_negotiation.cc:407] Connection header received
0806 01:19:04.197681 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:04.197689 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:04.197782 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:04.197906 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:04.199556 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:04.200678 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:04.202301 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:04.202992 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:04.203158 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:04.206946 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:04.206951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:04.206955 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:04.206987 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:04.209886 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:04.210830 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:04.210838 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:04.210845 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:04.210995 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:04.211571 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:04.211579 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:04.211584 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:04.212011 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:04.212204 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:04.212976 (+ 772us) server_negotiation.cc:299] Negotiation successful
0806 01:19:04.213252 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":311,"threads_started":1}
I20260806 01:19:04.214808 22273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:04.194590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:40155 (local address 127.25.236.129:59449)
0806 01:19:04.195628 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:04.195676 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:04.197391 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:04.198098 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:04.198105 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:04.198788 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:19:04.199369 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:04.199387 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:04.200868 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:04.200872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:04.202135 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:04.202149 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:04.203216 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:04.203221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:04.203416 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:04.204112 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:19:04.204135 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:19:04.206724 (+ 2589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:19:04.210047 (+ 3323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:19:04.210055 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:19:04.210067 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:19:04.210691 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:19:04.211135 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:19:04.211139 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:19:04.211141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:19:04.211433 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:19:04.212202 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:19:04.212208 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:19:04.212555 (+ 347us) client_negotiation.cc:769] Sending connection context
0806 01:19:04.212800 (+ 245us) client_negotiation.cc:240] Negotiation successful
0806 01:19:04.213224 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":549,"threads_started":1}
I20260806 01:19:04.216636 22265 heartbeater.cc:347] Connected to a master server at 127.25.236.190:40155
I20260806 01:19:04.217028 22265 heartbeater.cc:464] Registering TS with master...
I20260806 01:19:04.217907 22265 heartbeater.cc:511] Master 127.25.236.190:40155 requested a full tablet report, sending...
I20260806 01:19:04.220693 21994 ts_manager.cc:194] Registered new tserver with Master: e1574685c3434c3386642e828394633b (127.25.236.129:45591)
I20260806 01:19:04.222920 21994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:59449
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:19:04.267928 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:40155
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:04.618358 22276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:04.618636 22276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:04.618853 22276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:04.626062 22276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:04.626168 22276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:04.626420 22276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:19:04.636387 22276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:40155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:04.637545 22276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:04.638948 22276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:04.652309 22282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:04.652263 22287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:04.652347 22285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:04.653216 22276 server_base.cc:1034] running on GCE node
I20260806 01:19:04.653820 22276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:04.654886 22276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:04.656121 22276 hybrid_clock.cc:648] HybridClock initialized: now 1785979144656037 us; error 77 us; skew 500 ppm
I20260806 01:19:04.656454 22276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:04 dist-test-slave-7nm7 krb5kdc[21927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979144, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:04.659672 22276 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:19:04.661187 22276 webserver.cc:460] Webserver started at http://127.25.236.130:41101/ using document root <none> and password file <none>
I20260806 01:19:04.661917 22276 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:04.662072 22276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:04.662477 22276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:04.665625 22276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "527e3c3f65a248799cc8f19f2b9a4207"
format_stamp: "Formatted at 2026-08-06 01:19:04 on dist-test-slave-7nm7"
server_key: "dae07efc57e5b9988449f2ebac300d55"
server_key_iv: "b76b4a2ac8b79beb6b6fcb5c3daf23db"
server_key_version: "encryptionkey@0"
I20260806 01:19:04.666478 22276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "527e3c3f65a248799cc8f19f2b9a4207"
format_stamp: "Formatted at 2026-08-06 01:19:04 on dist-test-slave-7nm7"
server_key: "dae07efc57e5b9988449f2ebac300d55"
server_key_iv: "b76b4a2ac8b79beb6b6fcb5c3daf23db"
server_key_version: "encryptionkey@0"
I20260806 01:19:04.671679 22276 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 01:19:04.675169 22301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:04.676692 22276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:19:04.676883 22276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "527e3c3f65a248799cc8f19f2b9a4207"
format_stamp: "Formatted at 2026-08-06 01:19:04 on dist-test-slave-7nm7"
server_key: "dae07efc57e5b9988449f2ebac300d55"
server_key_iv: "b76b4a2ac8b79beb6b6fcb5c3daf23db"
server_key_version: "encryptionkey@0"
I20260806 01:19:04.677100 22276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:04.690006 22276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:04.693955 22276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:04.694298 22276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:04.695261 22276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:19:04.696565 22276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:19:04.696671 22276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:04.696820 22276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:19:04.696899 22276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:04.731482 22276 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:35131
I20260806 01:19:04.731532 22463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:35131 every 8 connection(s)
I20260806 01:19:04.732795 22276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:19:04.740372 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 22276
I20260806 01:19:04.740587 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:19:04.741163 26546 external_mini_cluster.cc:1428] Setting key f0ca54d67dcf93b2ae63d8c1861a277f
Aug 06 01:19:04 dist-test-slave-7nm7 krb5kdc[21927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979144, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:19:04.756222 22466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:04.735581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:53315 (local address 127.25.236.190:40155)
0806 01:19:04.736294 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:04.736301 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:04.737524 (+ 1223us) server_negotiation.cc:407] Connection header received
0806 01:19:04.739394 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:04.739399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:04.739467 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:04.739574 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:04.741532 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:04.742532 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:04.744945 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:04.745412 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:04.745517 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:04.749812 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:04.749817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:04.749821 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:04.749854 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:04.752776 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:04.753555 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:04.753566 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:04.753569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:04.753744 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:04.754282 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:04.754288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:04.754291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:04.754577 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:04.754763 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:04.755611 (+ 848us) server_negotiation.cc:299] Negotiation successful
0806 01:19:04.755853 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":390,"threads_started":1}
I20260806 01:19:04.757205 22468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:04.736571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:40155 (local address 127.25.236.130:53315)
0806 01:19:04.737346 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:04.737385 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:04.739126 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:04.739762 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:04.739769 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:04.740538 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:19:04.741346 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:04.741373 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:04.742716 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:04.742720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:04.744776 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:04.744796 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:04.745662 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:04.745671 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:04.746005 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:04.746800 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:19:04.746835 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:19:04.749575 (+ 2740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:19:04.752943 (+ 3368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:19:04.752955 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:19:04.752972 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:19:04.753413 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:19:04.753865 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:19:04.753870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:19:04.753873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:19:04.754117 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:19:04.754721 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:19:04.754731 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:19:04.755080 (+ 349us) client_negotiation.cc:769] Sending connection context
0806 01:19:04.755360 (+ 280us) client_negotiation.cc:240] Negotiation successful
0806 01:19:04.755886 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":314,"threads_started":1}
I20260806 01:19:04.758857 22464 heartbeater.cc:347] Connected to a master server at 127.25.236.190:40155
I20260806 01:19:04.759208 22464 heartbeater.cc:464] Registering TS with master...
I20260806 01:19:04.760069 22464 heartbeater.cc:511] Master 127.25.236.190:40155 requested a full tablet report, sending...
I20260806 01:19:04.761816 21994 ts_manager.cc:194] Registered new tserver with Master: 527e3c3f65a248799cc8f19f2b9a4207 (127.25.236.130:35131)
I20260806 01:19:04.762836 21994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:53315
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:19:04.813999 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:40155
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:05.172891 22473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:05.173178 22473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:05.173277 22473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:05.181591 22473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:05.181723 22473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:05.181926 22473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:19:05.195091 22473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:40155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:05.196652 22473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:05.197908 22473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:05.211210 22478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:05.211297 22483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:05.211241 22481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:05.212491 22473 server_base.cc:1034] running on GCE node
I20260806 01:19:05.213100 22473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:05.214223 22473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:05.215523 22473 hybrid_clock.cc:648] HybridClock initialized: now 1785979145215482 us; error 31 us; skew 500 ppm
I20260806 01:19:05.215867 22473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:05 dist-test-slave-7nm7 krb5kdc[21927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979145, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:05.219480 22473 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:19:05.221009 22473 webserver.cc:460] Webserver started at http://127.25.236.131:42523/ using document root <none> and password file <none>
I20260806 01:19:05.221743 22473 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:05.221899 22473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:05.222390 22473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:05.225672 22473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "6d51c27aa0c64430825cfb00bf44aa65"
format_stamp: "Formatted at 2026-08-06 01:19:05 on dist-test-slave-7nm7"
server_key: "bb6446b8c8f6d1b5496198d72b40c031"
server_key_iv: "9d2db7c6f928f414bcdb3e692b620468"
server_key_version: "encryptionkey@0"
I20260806 01:19:05.226547 22473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "6d51c27aa0c64430825cfb00bf44aa65"
format_stamp: "Formatted at 2026-08-06 01:19:05 on dist-test-slave-7nm7"
server_key: "bb6446b8c8f6d1b5496198d72b40c031"
server_key_iv: "9d2db7c6f928f414bcdb3e692b620468"
server_key_version: "encryptionkey@0"
I20260806 01:19:05.226928 22265 heartbeater.cc:503] Master 127.25.236.190:40155 was elected leader, sending a full tablet report...
I20260806 01:19:05.234163 22473 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 01:19:05.237751 22492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:05.239074 22473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:05.239264 22473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "6d51c27aa0c64430825cfb00bf44aa65"
format_stamp: "Formatted at 2026-08-06 01:19:05 on dist-test-slave-7nm7"
server_key: "bb6446b8c8f6d1b5496198d72b40c031"
server_key_iv: "9d2db7c6f928f414bcdb3e692b620468"
server_key_version: "encryptionkey@0"
I20260806 01:19:05.239461 22473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:05.253604 22473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:05.257990 22473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:05.258380 22473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:05.259359 22473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:19:05.260733 22473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:19:05.260840 22473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:05.260973 22473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:19:05.261051 22473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:05.298982 22473 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:46711
I20260806 01:19:05.299002 22652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:46711 every 8 connection(s)
I20260806 01:19:05.300827 22473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:19:05.308236 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 22473
I20260806 01:19:05.308454 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:19:05.308940 26546 external_mini_cluster.cc:1428] Setting key 914e6c92e2dcfb9f634bb2fd016aea1b
Aug 06 01:19:05 dist-test-slave-7nm7 krb5kdc[21927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979145, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:19:05.321759 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.303473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:46449 (local address 127.25.236.190:40155)
0806 01:19:05.303926 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:05.303932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:05.304915 (+ 983us) server_negotiation.cc:407] Connection header received
0806 01:19:05.306772 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:05.306776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:05.306865 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:05.306967 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:05.308681 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.309813 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.311999 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.312392 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.312490 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.315936 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:05.315941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:05.315944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:05.315977 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:05.319002 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:05.319675 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:05.319678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:05.319681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:05.319792 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:05.320242 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:05.320246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:05.320248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:05.320536 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:05.320667 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:05.321320 (+ 653us) server_negotiation.cc:299] Negotiation successful
0806 01:19:05.321488 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":210,"threads_started":1}
I20260806 01:19:05.322770 22657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.304005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:40155 (local address 127.25.236.131:46449)
0806 01:19:05.304747 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:05.304797 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:05.306526 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:05.307166 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:05.307173 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:05.307693 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:19:05.308495 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:05.308523 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.309994 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.309998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:05.311840 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:05.311852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.312605 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.312610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:05.312820 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.313521 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:19:05.313547 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:19:05.315737 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:19:05.319157 (+ 3420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:19:05.319164 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:19:05.319174 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:19:05.319567 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:19:05.319882 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:19:05.319885 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:19:05.319887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:19:05.320056 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:19:05.320670 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:19:05.320675 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:19:05.320958 (+ 283us) client_negotiation.cc:769] Sending connection context
0806 01:19:05.321172 (+ 214us) client_negotiation.cc:240] Negotiation successful
0806 01:19:05.321535 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":296,"threads_started":1}
I20260806 01:19:05.324242 22653 heartbeater.cc:347] Connected to a master server at 127.25.236.190:40155
I20260806 01:19:05.324594 22653 heartbeater.cc:464] Registering TS with master...
I20260806 01:19:05.325341 22653 heartbeater.cc:511] Master 127.25.236.190:40155 requested a full tablet report, sending...
I20260806 01:19:05.326861 21993 ts_manager.cc:194] Registered new tserver with Master: 6d51c27aa0c64430825cfb00bf44aa65 (127.25.236.131:46711)
I20260806 01:19:05.327767 21993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:46449
I20260806 01:19:05.337097 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:19:05.355046 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.341626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37206 (local address 127.25.236.190:40155)
0806 01:19:05.341876 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:05.341885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:05.342292 (+ 407us) server_negotiation.cc:407] Connection header received
0806 01:19:05.342518 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:05.342522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:05.342598 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:05.342718 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:05.344398 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.345151 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.346584 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.346988 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.347097 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.349748 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:05.349753 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:05.349756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:05.349788 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:05.352245 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:05.352941 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:05.352945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:05.352948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:05.353092 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:05.353577 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:05.353580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:05.353583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:05.353875 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:05.354144 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:05.354558 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 01:19:05.354726 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 01:19:05.358862 21993 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:19:05.365936 21993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:19:05.405063 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.378576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:45591 (local address 127.0.0.1:40068)
0806 01:19:05.385652 (+ 7076us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:05.385712 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:05.385964 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:05.398289 (+ 12325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:05.398297 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:05.398332 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:05.398707 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:05.398718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.400548 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.400553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:05.402432 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:05.402447 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.403954 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.403961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:05.404137 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.404209 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 01:19:05.404352 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 01:19:05.404548 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6694,"mutex_wait_us":26,"thread_start_us":395,"threads_started":1}
I20260806 01:19:05.405642 22678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.379296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40068 (local address 127.25.236.129:45591)
0806 01:19:05.397691 (+ 18395us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:05.397699 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:05.397723 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:19:05.397834 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:05.397841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:05.398041 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:05.398892 (+ 851us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:05.399100 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.400399 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.402625 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.403739 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.403961 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.404143 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:05.404410 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 01:19:05.404575 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18207,"thread_start_us":18013,"threads_started":1}
I20260806 01:19:05.405906 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.378628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:46711 (local address 127.0.0.1:44698)
0806 01:19:05.395818 (+ 17190us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:05.395843 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:05.396003 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:05.397511 (+ 1508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:05.397518 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:05.397552 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:05.397942 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:05.397954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.400025 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.400029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:05.402888 (+ 2859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:05.402903 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.405011 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.405018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:05.405240 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.405274 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:19:05.405365 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 01:19:05.405492 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17002,"mutex_wait_us":261,"thread_start_us":17485,"threads_started":1}
I20260806 01:19:05.406373 22679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.379781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:35131 (local address 127.0.0.1:57480)
0806 01:19:05.397289 (+ 17508us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:05.397310 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:05.397437 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:05.398073 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:05.398079 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:05.398107 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:05.398640 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:05.398660 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.399961 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.399969 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:05.402891 (+ 2922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:05.402903 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.405026 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.405031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:05.405488 (+ 457us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.405517 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 01:19:05.405608 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 01:19:05.405723 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17364,"mutex_wait_us":220,"thread_start_us":17221,"threads_started":1}
I20260806 01:19:05.406589 22680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.380311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57480 (local address 127.25.236.130:35131)
0806 01:19:05.396804 (+ 16493us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:05.396816 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:05.397566 (+ 750us) server_negotiation.cc:407] Connection header received
0806 01:19:05.397670 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:05.397676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:05.397880 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:05.398080 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:05.398839 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.399821 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.403077 (+ 3256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.404490 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.404648 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.404817 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:05.406014 (+ 1197us) server_negotiation.cc:299] Negotiation successful
0806 01:19:05.406202 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16325,"thread_start_us":16174,"threads_started":1}
I20260806 01:19:05.407011 22677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.378727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44698 (local address 127.25.236.131:46711)
0806 01:19:05.395760 (+ 17033us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:05.395770 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:05.396474 (+ 704us) server_negotiation.cc:407] Connection header received
0806 01:19:05.396596 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:05.396605 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:05.396899 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:05.397143 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:05.398175 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.399843 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.403094 (+ 3251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.404484 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.404673 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.404823 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:05.406276 (+ 1453us) server_negotiation.cc:299] Negotiation successful
0806 01:19:05.406592 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16827,"thread_start_us":16652,"threads_started":1}
I20260806 01:19:05.409929 22368 tablet_service.cc:1467] Processing CreateTablet for tablet f8576ba502ed4668b430df58be896a9f (DEFAULT_TABLE table=test-table [id=f27e8ed0211e487e9ba9cd385b9433a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:05.410017 22567 tablet_service.cc:1467] Processing CreateTablet for tablet f8576ba502ed4668b430df58be896a9f (DEFAULT_TABLE table=test-table [id=f27e8ed0211e487e9ba9cd385b9433a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:05.409921 22176 tablet_service.cc:1467] Processing CreateTablet for tablet f8576ba502ed4668b430df58be896a9f (DEFAULT_TABLE table=test-table [id=f27e8ed0211e487e9ba9cd385b9433a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:05.411262 22368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8576ba502ed4668b430df58be896a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:05.411386 22176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8576ba502ed4668b430df58be896a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:05.411386 22567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8576ba502ed4668b430df58be896a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:05.422969 22681 tablet_bootstrap.cc:492] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207: Bootstrap starting.
I20260806 01:19:05.425608 22682 tablet_bootstrap.cc:492] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b: Bootstrap starting.
I20260806 01:19:05.426262 22681 tablet_bootstrap.cc:654] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:05.427659 22681 log.cc:824] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:05.429157 22682 tablet_bootstrap.cc:654] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:05.430444 22682 log.cc:824] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:05.430655 22683 tablet_bootstrap.cc:492] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65: Bootstrap starting.
I20260806 01:19:05.431864 22681 tablet_bootstrap.cc:492] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207: No bootstrap required, opened a new log
I20260806 01:19:05.432137 22681 ts_tablet_manager.cc:1397] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260806 01:19:05.434784 22683 tablet_bootstrap.cc:654] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:05.436045 22683 log.cc:824] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:05.436952 22682 tablet_bootstrap.cc:492] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b: No bootstrap required, opened a new log
I20260806 01:19:05.437238 22682 ts_tablet_manager.cc:1397] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260806 01:19:05.439244 22683 tablet_bootstrap.cc:492] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65: No bootstrap required, opened a new log
I20260806 01:19:05.439491 22683 ts_tablet_manager.cc:1397] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260806 01:19:05.440961 22681 raft_consensus.cc:348] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d51c27aa0c64430825cfb00bf44aa65" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46711 } } peers { permanent_uuid: "e1574685c3434c3386642e828394633b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45591 } } peers { permanent_uuid: "527e3c3f65a248799cc8f19f2b9a4207" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35131 } }
I20260806 01:19:05.441332 22681 raft_consensus.cc:374] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:05.441432 22681 raft_consensus.cc:729] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 527e3c3f65a248799cc8f19f2b9a4207, State: Initialized, Role: FOLLOWER
I20260806 01:19:05.442076 22681 consensus_queue.cc:260] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d51c27aa0c64430825cfb00bf44aa65" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46711 } } peers { permanent_uuid: "e1574685c3434c3386642e828394633b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45591 } } peers { permanent_uuid: "527e3c3f65a248799cc8f19f2b9a4207" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35131 } }
I20260806 01:19:05.447075 22464 heartbeater.cc:503] Master 127.25.236.190:40155 was elected leader, sending a full tablet report...
I20260806 01:19:05.447860 22681 ts_tablet_manager.cc:1428] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260806 01:19:05.449287 22682 raft_consensus.cc:348] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d51c27aa0c64430825cfb00bf44aa65" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46711 } } peers { permanent_uuid: "e1574685c3434c3386642e828394633b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45591 } } peers { permanent_uuid: "527e3c3f65a248799cc8f19f2b9a4207" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35131 } }
I20260806 01:19:05.449745 22682 raft_consensus.cc:374] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:05.449862 22682 raft_consensus.cc:729] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1574685c3434c3386642e828394633b, State: Initialized, Role: FOLLOWER
I20260806 01:19:05.450539 22682 consensus_queue.cc:260] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d51c27aa0c64430825cfb00bf44aa65" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46711 } } peers { permanent_uuid: "e1574685c3434c3386642e828394633b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45591 } } peers { permanent_uuid: "527e3c3f65a248799cc8f19f2b9a4207" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35131 } }
I20260806 01:19:05.451987 22683 raft_consensus.cc:348] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d51c27aa0c64430825cfb00bf44aa65" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46711 } } peers { permanent_uuid: "e1574685c3434c3386642e828394633b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45591 } } peers { permanent_uuid: "527e3c3f65a248799cc8f19f2b9a4207" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35131 } }
I20260806 01:19:05.452615 22683 raft_consensus.cc:374] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:05.452752 22683 raft_consensus.cc:729] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d51c27aa0c64430825cfb00bf44aa65, State: Initialized, Role: FOLLOWER
I20260806 01:19:05.453672 22683 consensus_queue.cc:260] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d51c27aa0c64430825cfb00bf44aa65" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46711 } } peers { permanent_uuid: "e1574685c3434c3386642e828394633b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45591 } } peers { permanent_uuid: "527e3c3f65a248799cc8f19f2b9a4207" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35131 } }
I20260806 01:19:05.454953 22653 heartbeater.cc:503] Master 127.25.236.190:40155 was elected leader, sending a full tablet report...
I20260806 01:19:05.456043 22683 ts_tablet_manager.cc:1428] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
I20260806 01:19:05.457628 22682 ts_tablet_manager.cc:1428] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b: Time spent starting tablet: real 0.020s user 0.017s sys 0.004s
W20260806 01:19:05.484990 22465 tablet.cc:2369] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:19:05.552618 22654 tablet.cc:2369] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:19:05.582628 22694 raft_consensus.cc:482] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:19:05.583020 22694 raft_consensus.cc:504] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d51c27aa0c64430825cfb00bf44aa65" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46711 } } peers { permanent_uuid: "e1574685c3434c3386642e828394633b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45591 } } peers { permanent_uuid: "527e3c3f65a248799cc8f19f2b9a4207" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35131 } }
I20260806 01:19:05.584828 22694 leader_election.cc:288] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1574685c3434c3386642e828394633b (127.25.236.129:45591), 527e3c3f65a248799cc8f19f2b9a4207 (127.25.236.130:35131)
I20260806 01:19:05.590642 22678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.585422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:35391 (local address 127.25.236.129:45591)
0806 01:19:05.585637 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:05.585646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:05.585669 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:19:05.585764 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:05.585773 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:05.585871 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:05.585970 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:05.586555 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.587305 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.588900 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.589690 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.589858 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.589936 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:05.590225 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 01:19:05.590382 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:19:05.590631 22657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.585096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:45591 (local address 127.25.236.131:35391)
0806 01:19:05.585311 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:05.585327 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:05.585476 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:05.586154 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:05.586158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:05.586182 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:05.586410 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:05.586418 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.587437 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.587446 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:05.588736 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:05.588747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.589845 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.589851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:05.590037 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.590062 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 01:19:05.590189 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 01:19:05.590345 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 01:19:05.591558 22700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.585515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:35131 (local address 127.25.236.131:54219)
0806 01:19:05.586154 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:05.586169 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:05.586347 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:05.586677 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:05.586681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:05.586704 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:05.587025 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:05.587033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.588018 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.588023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:05.589546 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:05.589558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.590782 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.590788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:05.590989 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.591028 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 01:19:05.591152 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 01:19:05.591254 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"spinlock_wait_cycles":4480,"thread_start_us":311,"threads_started":1}
I20260806 01:19:05.591676 22680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.585800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:54219 (local address 127.25.236.130:35131)
0806 01:19:05.586027 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:05.586033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:05.586266 (+ 233us) server_negotiation.cc:407] Connection header received
0806 01:19:05.586491 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:05.586498 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:05.586563 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:05.586661 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:05.587170 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.587863 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.589716 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.590581 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.590757 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.590829 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:05.591228 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 01:19:05.591393 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 01:19:05.592079 22202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8576ba502ed4668b430df58be896a9f" candidate_uuid: "6d51c27aa0c64430825cfb00bf44aa65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1574685c3434c3386642e828394633b" is_pre_election: true
I20260806 01:19:05.592717 22202 raft_consensus.cc:2454] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d51c27aa0c64430825cfb00bf44aa65 in term 0.
I20260806 01:19:05.592662 22395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8576ba502ed4668b430df58be896a9f" candidate_uuid: "6d51c27aa0c64430825cfb00bf44aa65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "527e3c3f65a248799cc8f19f2b9a4207" is_pre_election: true
I20260806 01:19:05.593215 22395 raft_consensus.cc:2454] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d51c27aa0c64430825cfb00bf44aa65 in term 0.
I20260806 01:19:05.593463 22499 leader_election.cc:302] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d51c27aa0c64430825cfb00bf44aa65, e1574685c3434c3386642e828394633b; no voters:
I20260806 01:19:05.593896 22694 raft_consensus.cc:2781] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:19:05.594053 22694 raft_consensus.cc:482] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:19:05.594195 22694 raft_consensus.cc:3037] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:05.596722 22694 raft_consensus.cc:504] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d51c27aa0c64430825cfb00bf44aa65" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46711 } } peers { permanent_uuid: "e1574685c3434c3386642e828394633b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45591 } } peers { permanent_uuid: "527e3c3f65a248799cc8f19f2b9a4207" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35131 } }
I20260806 01:19:05.597352 22694 leader_election.cc:288] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [CANDIDATE]: Term 1 election: Requested vote from peers e1574685c3434c3386642e828394633b (127.25.236.129:45591), 527e3c3f65a248799cc8f19f2b9a4207 (127.25.236.130:35131)
I20260806 01:19:05.597851 22202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8576ba502ed4668b430df58be896a9f" candidate_uuid: "6d51c27aa0c64430825cfb00bf44aa65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1574685c3434c3386642e828394633b"
I20260806 01:19:05.597939 22395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8576ba502ed4668b430df58be896a9f" candidate_uuid: "6d51c27aa0c64430825cfb00bf44aa65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "527e3c3f65a248799cc8f19f2b9a4207"
I20260806 01:19:05.598078 22202 raft_consensus.cc:3037] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:05.598130 22395 raft_consensus.cc:3037] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:05.601338 22202 raft_consensus.cc:2454] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d51c27aa0c64430825cfb00bf44aa65 in term 1.
I20260806 01:19:05.601338 22395 raft_consensus.cc:2454] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d51c27aa0c64430825cfb00bf44aa65 in term 1.
I20260806 01:19:05.601738 22499 leader_election.cc:302] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d51c27aa0c64430825cfb00bf44aa65, e1574685c3434c3386642e828394633b; no voters:
I20260806 01:19:05.602008 22694 raft_consensus.cc:2781] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:05.602540 22694 raft_consensus.cc:686] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [term 1 LEADER]: Becoming Leader. State: Replica: 6d51c27aa0c64430825cfb00bf44aa65, State: Running, Role: LEADER
I20260806 01:19:05.603111 22694 consensus_queue.cc:237] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d51c27aa0c64430825cfb00bf44aa65" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46711 } } peers { permanent_uuid: "e1574685c3434c3386642e828394633b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45591 } } peers { permanent_uuid: "527e3c3f65a248799cc8f19f2b9a4207" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35131 } }
I20260806 01:19:05.607798 21994 catalog_manager.cc:5495] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d51c27aa0c64430825cfb00bf44aa65 (127.25.236.131). New cstate: current_term: 1 leader_uuid: "6d51c27aa0c64430825cfb00bf44aa65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d51c27aa0c64430825cfb00bf44aa65" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1574685c3434c3386642e828394633b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "527e3c3f65a248799cc8f19f2b9a4207" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35131 } health_report { overall_health: UNKNOWN } } }
I20260806 01:19:05.631240 22677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.625903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44714 (local address 127.25.236.131:46711)
0806 01:19:05.626117 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:05.626122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:05.626141 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:19:05.626344 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:05.626348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:05.626428 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:05.626558 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:19:05.626952 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.627686 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.629042 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.629422 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.629539 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.630030 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:19:05.630599 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:19:05.630729 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:05.630878 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 01:19:05.630944 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 01:19:05.639727 22395 raft_consensus.cc:1261] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207 [term 1 FOLLOWER]: Refusing update from remote peer 6d51c27aa0c64430825cfb00bf44aa65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:19:05.639710 22202 raft_consensus.cc:1261] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b [term 1 FOLLOWER]: Refusing update from remote peer 6d51c27aa0c64430825cfb00bf44aa65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:19:05.640755 22701 consensus_queue.cc:1035] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1574685c3434c3386642e828394633b" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:19:05.641222 22694 consensus_queue.cc:1035] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "527e3c3f65a248799cc8f19f2b9a4207" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:19:05.659060 22706 mvcc.cc:205] Tried to move back new op lower bound from 7315370580528631808 to 7315370580394315776. Current Snapshot: MvccSnapshot[applied={T|T < 7315370580528631808}]
I20260806 01:19:05.671659 21994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:19:05.671941 21994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:19:05.675654 21994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae9f30c69e7b446b91b8d7c14747f603: Sending DeleteTablet for 3 replicas of tablet f8576ba502ed4668b430df58be896a9f
I20260806 01:19:05.677024 22176 tablet_service.cc:1514] Processing DeleteTablet for tablet f8576ba502ed4668b430df58be896a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:05 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:40068
I20260806 01:19:05.677129 22567 tablet_service.cc:1514] Processing DeleteTablet for tablet f8576ba502ed4668b430df58be896a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:05 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:44698
I20260806 01:19:05.677294 22368 tablet_service.cc:1514] Processing DeleteTablet for tablet f8576ba502ed4668b430df58be896a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:05 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:57480
I20260806 01:19:05.679001 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 22082
I20260806 01:19:05.696070 22718 tablet_replica.cc:330] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207: stopping tablet replica
I20260806 01:19:05.696254 22715 tablet_replica.cc:330] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b: stopping tablet replica
I20260806 01:19:05.696589 22715 raft_consensus.cc:2229] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:19:05.696589 22718 raft_consensus.cc:2229] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:19:05.697104 22715 raft_consensus.cc:2258] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:05.697135 22718 raft_consensus.cc:2258] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:05.699420 22717 tablet_replica.cc:330] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65: stopping tablet replica
I20260806 01:19:05.699888 22717 raft_consensus.cc:2229] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:19:05.700443 22715 ts_tablet_manager.cc:1905] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:05.701115 22717 raft_consensus.cc:2258] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 06 01:19:05 dist-test-slave-7nm7 krb5kdc[21927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:19:05.703815 22718 ts_tablet_manager.cc:1905] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:05.708477 22715 ts_tablet_manager.cc:1918] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:05.708767 22715 log.cc:1196] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/f8576ba502ed4668b430df58be896a9f
I20260806 01:19:05.709424 22715 ts_tablet_manager.cc:1939] T f8576ba502ed4668b430df58be896a9f P e1574685c3434c3386642e828394633b: Deleting consensus metadata
I20260806 01:19:05.710453 22717 ts_tablet_manager.cc:1905] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:05.711694 21973 catalog_manager.cc:4841] TS e1574685c3434c3386642e828394633b (127.25.236.129:45591): tablet f8576ba502ed4668b430df58be896a9f (table test-table [id=f27e8ed0211e487e9ba9cd385b9433a8]) successfully deleted
I20260806 01:19:05.715279 22678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.680811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40070 (local address 127.25.236.129:45591)
0806 01:19:05.681038 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:05.681047 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:05.685937 (+ 4890us) server_negotiation.cc:407] Connection header received
0806 01:19:05.686839 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:05.686849 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:05.686960 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:05.687083 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:05.691337 (+ 4254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.692953 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.697502 (+ 4549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.698427 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.698589 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.705256 (+ 6667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:05.705272 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:05.705294 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:05.705341 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:05.710128 (+ 4787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:05.711191 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:05.711201 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:05.711204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:05.712067 (+ 863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:05.713002 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:05.713009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:05.713015 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:05.713515 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:05.713865 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:05.714596 (+ 731us) server_negotiation.cc:299] Negotiation successful
0806 01:19:05.714871 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:19:05.716481 22258 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:05.721971 22718 ts_tablet_manager.cc:1918] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:05.722182 22718 log.cc:1196] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/f8576ba502ed4668b430df58be896a9f
I20260806 01:19:05.722781 22718 ts_tablet_manager.cc:1939] T f8576ba502ed4668b430df58be896a9f P 527e3c3f65a248799cc8f19f2b9a4207: Deleting consensus metadata
I20260806 01:19:05.724830 22717 ts_tablet_manager.cc:1918] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:05.725019 22717 log.cc:1196] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/f8576ba502ed4668b430df58be896a9f
I20260806 01:19:05.725152 21973 catalog_manager.cc:4841] TS 527e3c3f65a248799cc8f19f2b9a4207 (127.25.236.130:35131): tablet f8576ba502ed4668b430df58be896a9f (table test-table [id=f27e8ed0211e487e9ba9cd385b9433a8]) successfully deleted
I20260806 01:19:05.725775 22717 ts_tablet_manager.cc:1939] T f8576ba502ed4668b430df58be896a9f P 6d51c27aa0c64430825cfb00bf44aa65: Deleting consensus metadata
I20260806 01:19:05.727763 21972 catalog_manager.cc:4841] TS 6d51c27aa0c64430825cfb00bf44aa65 (127.25.236.131:46711): tablet f8576ba502ed4668b430df58be896a9f (table test-table [id=f27e8ed0211e487e9ba9cd385b9433a8]) successfully deleted
I20260806 01:19:05.832700 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 22082
I20260806 01:19:05.862108 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 22276
Aug 06 01:19:05 dist-test-slave-7nm7 krb5kdc[21927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:19:05.880689 22680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:05.863057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57488 (local address 127.25.236.130:35131)
0806 01:19:05.863255 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:05.863261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:05.863311 (+ 50us) server_negotiation.cc:407] Connection header received
0806 01:19:05.863517 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:05.863523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:05.863608 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:05.863731 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:05.865146 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.866406 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.867972 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:05.868597 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:05.868745 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:05.873372 (+ 4627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:05.873391 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:05.873412 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:05.873461 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:05.877679 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:05.878444 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:05.878454 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:05.878458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:05.878617 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:05.878949 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:05.878954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:05.878958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:05.879444 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:05.879727 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:05.879991 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 01:19:05.880280 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:19:05.881433 22456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:05.976099 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 22276
I20260806 01:19:06.001828 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 22473
Aug 06 01:19:06 dist-test-slave-7nm7 krb5kdc[21927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:19:06.022670 22677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:06.002888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44724 (local address 127.25.236.131:46711)
0806 01:19:06.003046 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:06.003051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:06.003069 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:19:06.003280 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:06.003284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:06.003366 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:06.003446 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:06.005483 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:06.006669 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:06.008609 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:06.009069 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:06.009200 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:06.015867 (+ 6667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:06.015883 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:06.015901 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:06.015937 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:06.019690 (+ 3753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:06.020457 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:06.020465 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:06.020467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:06.020623 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:06.021039 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:06.021049 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:06.021052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:06.021446 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:06.021795 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:06.022044 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 01:19:06.022302 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 01:19:06.023418 22648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:06.145104 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 22473
I20260806 01:19:06.177341 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 21946
I20260806 01:19:06.178572 22036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:06.293407 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 21946
2026-08-06T01:19:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4992 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T01:19:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:06Z Disabled control of system clock
I20260806 01:19:06.395674 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:33945
--webserver_interface=127.25.236.190
--webserver_port=0
--builtin_ntp_servers=127.25.236.148:45349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:33945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:19:06.760099 22731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:06.760694 22731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:06.760854 22731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:06.768365 22731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:06.768472 22731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:06.768579 22731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:06.768692 22731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:06.768775 22731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:06.778266 22731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:33945
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:33945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:06.779867 22731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:06.781838 22731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:06.794045 22738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:06.793974 22739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:06.794148 22741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:06.795370 22731 server_base.cc:1034] running on GCE node
I20260806 01:19:06.796324 22731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:06.797854 22731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:06.799055 22731 hybrid_clock.cc:648] HybridClock initialized: now 1785979146798966 us; error 85 us; skew 500 ppm
I20260806 01:19:06.799515 22731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:19:06.801782 22731 webserver.cc:460] Webserver started at http://127.25.236.190:41559/ using document root <none> and password file <none>
I20260806 01:19:06.802527 22731 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:06.802624 22731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:06.802894 22731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:06.805720 22731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "65fae9f02af24a1ea512ba91f55998cb"
format_stamp: "Formatted at 2026-08-06 01:19:06 on dist-test-slave-7nm7"
server_key: "b7e0b81ff1b49abecccacdddbe502d24"
server_key_iv: "3853c0aa1be5a1aa30d2b3b9d0bf1ce3"
server_key_version: "encryptionkey@0"
I20260806 01:19:06.806685 22731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "65fae9f02af24a1ea512ba91f55998cb"
format_stamp: "Formatted at 2026-08-06 01:19:06 on dist-test-slave-7nm7"
server_key: "b7e0b81ff1b49abecccacdddbe502d24"
server_key_iv: "3853c0aa1be5a1aa30d2b3b9d0bf1ce3"
server_key_version: "encryptionkey@0"
I20260806 01:19:06.811712 22731 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:19:06.815517 22748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:06.816944 22731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:06.817102 22731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "65fae9f02af24a1ea512ba91f55998cb"
format_stamp: "Formatted at 2026-08-06 01:19:06 on dist-test-slave-7nm7"
server_key: "b7e0b81ff1b49abecccacdddbe502d24"
server_key_iv: "3853c0aa1be5a1aa30d2b3b9d0bf1ce3"
server_key_version: "encryptionkey@0"
I20260806 01:19:06.817327 22731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:06.833073 22731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:06.833873 22731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:06.834157 22731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:06.853933 22731 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:33945
I20260806 01:19:06.853969 22821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:33945 every 8 connection(s)
I20260806 01:19:06.855386 22731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:06.858417 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 22731
I20260806 01:19:06.858609 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:06.859185 26546 external_mini_cluster.cc:1428] Setting key 9dca9235db9eb094e6e0e7f7947a070e
I20260806 01:19:06.860178 22822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:06.874982 22822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb: Bootstrap starting.
I20260806 01:19:06.875200 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:06.861366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46704 (local address 127.25.236.190:33945)
0806 01:19:06.862620 (+ 1254us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:06.862647 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:06.862702 (+ 55us) server_negotiation.cc:407] Connection header received
0806 01:19:06.865257 (+ 2555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:06.865281 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:06.865751 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:06.866112 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:06.866895 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:06.868527 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:06.870450 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:06.871160 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:06.871313 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:06.872682 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:06.872692 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:06.872711 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:06.873309 (+ 598us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:06.873392 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:06.873414 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:06.873513 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:06.873767 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 01:19:06.874139 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":462,"threads_started":1}
I20260806 01:19:06.878582 22822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:06.879592 22822 log.cc:824] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:06.882562 22822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb: No bootstrap required, opened a new log
I20260806 01:19:06.890101 22822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65fae9f02af24a1ea512ba91f55998cb" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33945 } }
I20260806 01:19:06.890554 22822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:06.890661 22822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65fae9f02af24a1ea512ba91f55998cb, State: Initialized, Role: FOLLOWER
I20260806 01:19:06.891309 22822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65fae9f02af24a1ea512ba91f55998cb" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33945 } }
I20260806 01:19:06.891577 22822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:06.891739 22822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:06.891963 22822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:06.894210 22822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65fae9f02af24a1ea512ba91f55998cb" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33945 } }
I20260806 01:19:06.894827 22822 leader_election.cc:302] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65fae9f02af24a1ea512ba91f55998cb; no voters:
I20260806 01:19:06.895489 22822 leader_election.cc:288] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:06.895661 22830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:06.897032 22830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [term 1 LEADER]: Becoming Leader. State: Replica: 65fae9f02af24a1ea512ba91f55998cb, State: Running, Role: LEADER
I20260806 01:19:06.897827 22822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:06.897725 22830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65fae9f02af24a1ea512ba91f55998cb" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33945 } }
I20260806 01:19:06.900970 22832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65fae9f02af24a1ea512ba91f55998cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65fae9f02af24a1ea512ba91f55998cb" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33945 } } }
I20260806 01:19:06.901038 22833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65fae9f02af24a1ea512ba91f55998cb. Latest consensus state: current_term: 1 leader_uuid: "65fae9f02af24a1ea512ba91f55998cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65fae9f02af24a1ea512ba91f55998cb" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 33945 } } }
I20260806 01:19:06.901422 22832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:06.901463 22833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:06.907020 22841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:06.912885 22841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:06.923990 22841 catalog_manager.cc:1347] Generated new cluster ID: a919b285572948b69630bcdb9d4e84a7
I20260806 01:19:06.924187 22841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:06.939221 22841 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:06.940771 22841 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:06.951243 22841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb: Generated new TSK 0
I20260806 01:19:06.952128 22841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:06.959636 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33945
--builtin_ntp_servers=127.25.236.148:45349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:19:07.329599 22862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:07.329927 22862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:07.330080 22862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:07.337442 22862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:07.337579 22862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:07.337738 22862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:19:07.346966 22862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:07.348225 22862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:07.349871 22862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:07.363000 22869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:07.362924 22870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:07.362957 22873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:07.364198 22862 server_base.cc:1034] running on GCE node
I20260806 01:19:07.364995 22862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:07.366217 22862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:07.367568 22862 hybrid_clock.cc:648] HybridClock initialized: now 1785979147367500 us; error 55 us; skew 500 ppm
I20260806 01:19:07.367936 22862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:19:07.370518 22862 webserver.cc:460] Webserver started at http://127.25.236.129:39169/ using document root <none> and password file <none>
I20260806 01:19:07.371261 22862 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:07.371402 22862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:07.371738 22862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:07.375528 22862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "521bf1ef556a4abb92a82a112abdcd0f"
format_stamp: "Formatted at 2026-08-06 01:19:07 on dist-test-slave-7nm7"
server_key: "c28d7cff14a46e42cacfb6661dfe53c6"
server_key_iv: "1d92b040a0f8e38aad4a718b64771120"
server_key_version: "encryptionkey@0"
I20260806 01:19:07.376410 22862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "521bf1ef556a4abb92a82a112abdcd0f"
format_stamp: "Formatted at 2026-08-06 01:19:07 on dist-test-slave-7nm7"
server_key: "c28d7cff14a46e42cacfb6661dfe53c6"
server_key_iv: "1d92b040a0f8e38aad4a718b64771120"
server_key_version: "encryptionkey@0"
I20260806 01:19:07.381719 22862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 01:19:07.385360 22880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:07.386910 22862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 01:19:07.387109 22862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "521bf1ef556a4abb92a82a112abdcd0f"
format_stamp: "Formatted at 2026-08-06 01:19:07 on dist-test-slave-7nm7"
server_key: "c28d7cff14a46e42cacfb6661dfe53c6"
server_key_iv: "1d92b040a0f8e38aad4a718b64771120"
server_key_version: "encryptionkey@0"
I20260806 01:19:07.387349 22862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:07.407119 22862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:07.408089 22862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:07.408417 22862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:07.409446 22862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:19:07.411044 22862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:19:07.411137 22862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:07.411285 22862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:19:07.411382 22862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:07.446601 22862 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:38001
I20260806 01:19:07.446697 23042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:38001 every 8 connection(s)
I20260806 01:19:07.447994 22862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:19:07.456880 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 22862
I20260806 01:19:07.457094 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:19:07.457654 26546 external_mini_cluster.cc:1428] Setting key e8a756d53e8e4468e0e59c4c37d479ec
I20260806 01:19:07.461231 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33945
--builtin_ntp_servers=127.25.236.148:45349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:19:07.464090 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:07.450958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:43693 (local address 127.25.236.190:33945)
0806 01:19:07.451428 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:07.451444 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:07.453263 (+ 1819us) server_negotiation.cc:407] Connection header received
0806 01:19:07.455447 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:07.455451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:07.455533 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:07.455654 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:07.456816 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:07.457927 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:07.460427 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:07.461091 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:07.461265 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:07.463051 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:07.463061 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:07.463070 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:07.463096 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:07.463150 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:07.463172 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:07.463276 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:07.463512 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 01:19:07.463626 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":217,"threads_started":1}
I20260806 01:19:07.465004 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:07.451612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:33945 (local address 127.25.236.129:43693)
0806 01:19:07.453051 (+ 1439us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:07.453097 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:07.455139 (+ 2042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:07.455901 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:07.455909 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:07.455999 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:19:07.456621 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:07.456638 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:07.458128 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:07.458133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:07.460190 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:07.460219 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:07.461506 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:07.461516 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:07.461783 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:07.462653 (+ 870us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:19:07.462656 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:19:07.462699 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:19:07.462796 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:19:07.463286 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:19:07.463295 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:19:07.463302 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 01:19:07.463449 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 01:19:07.463799 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":414,"threads_started":1}
I20260806 01:19:07.466674 23043 heartbeater.cc:347] Connected to a master server at 127.25.236.190:33945
I20260806 01:19:07.467000 23043 heartbeater.cc:464] Registering TS with master...
I20260806 01:19:07.468016 23043 heartbeater.cc:511] Master 127.25.236.190:33945 requested a full tablet report, sending...
I20260806 01:19:07.470206 22772 ts_manager.cc:194] Registered new tserver with Master: 521bf1ef556a4abb92a82a112abdcd0f (127.25.236.129:38001)
I20260806 01:19:07.471935 22772 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.236.129:43693
W20260806 01:19:07.819849 23048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:07.820144 23048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:07.820235 23048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:07.827775 23048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:07.827876 23048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:07.828051 23048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:19:07.838291 23048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:07.839685 23048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:07.841492 23048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:07.855374 23055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:07.856005 23056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:07.856544 23058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:07.856856 23048 server_base.cc:1034] running on GCE node
I20260806 01:19:07.857457 23048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:07.858426 23048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:07.859686 23048 hybrid_clock.cc:648] HybridClock initialized: now 1785979147859629 us; error 44 us; skew 500 ppm
I20260806 01:19:07.860036 23048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:19:07.862706 23048 webserver.cc:460] Webserver started at http://127.25.236.130:43835/ using document root <none> and password file <none>
I20260806 01:19:07.863422 23048 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:07.863562 23048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:07.863898 23048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:07.867085 23048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "e4e5016d49a94d69a5a3e565e95cd8dc"
format_stamp: "Formatted at 2026-08-06 01:19:07 on dist-test-slave-7nm7"
server_key: "cab6f00620f177edf00a97bdb2e3623e"
server_key_iv: "804e33978b0399369c348b96bdea7646"
server_key_version: "encryptionkey@0"
I20260806 01:19:07.867928 23048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "e4e5016d49a94d69a5a3e565e95cd8dc"
format_stamp: "Formatted at 2026-08-06 01:19:07 on dist-test-slave-7nm7"
server_key: "cab6f00620f177edf00a97bdb2e3623e"
server_key_iv: "804e33978b0399369c348b96bdea7646"
server_key_version: "encryptionkey@0"
I20260806 01:19:07.873195 23048 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:19:07.877390 23066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:07.879014 23048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:07.879225 23048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "e4e5016d49a94d69a5a3e565e95cd8dc"
format_stamp: "Formatted at 2026-08-06 01:19:07 on dist-test-slave-7nm7"
server_key: "cab6f00620f177edf00a97bdb2e3623e"
server_key_iv: "804e33978b0399369c348b96bdea7646"
server_key_version: "encryptionkey@0"
I20260806 01:19:07.879469 23048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:07.908555 23048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:07.909411 23048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:07.909718 23048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:07.910970 23048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:19:07.912467 23048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:19:07.912591 23048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:07.912734 23048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:19:07.912825 23048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:07.953104 23048 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:43277
I20260806 01:19:07.953199 23243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:43277 every 8 connection(s)
I20260806 01:19:07.954460 23048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:19:07.959322 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 23048
I20260806 01:19:07.959527 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:19:07.960141 26546 external_mini_cluster.cc:1428] Setting key e09cda2c0adb5dc7da20bd9798c94814
I20260806 01:19:07.963714 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33945
--builtin_ntp_servers=127.25.236.148:45349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:19:07.969789 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:07.957214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:36895 (local address 127.25.236.190:33945)
0806 01:19:07.957450 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:07.957458 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:07.959197 (+ 1739us) server_negotiation.cc:407] Connection header received
0806 01:19:07.961647 (+ 2450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:07.961651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:07.961722 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:07.961811 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:07.963448 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:07.964319 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:07.966890 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:07.967344 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:07.967451 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:07.969051 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:07.969056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:07.969059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:07.969076 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:07.969114 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:07.969125 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:07.969192 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:07.969462 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 01:19:07.969551 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":73}
I20260806 01:19:07.971243 23248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:07.957860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:33945 (local address 127.25.236.130:36895)
0806 01:19:07.958907 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:07.958966 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:07.961245 (+ 2279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:07.962084 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:07.962191 (+ 107us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:07.962369 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:19:07.963241 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:07.963268 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:07.964483 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:07.964491 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:07.966726 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:07.966744 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:07.967572 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:07.967581 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:07.967898 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:07.968684 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:19:07.968689 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:19:07.968748 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:19:07.968845 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:19:07.969218 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:19:07.969230 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:19:07.969240 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 01:19:07.969428 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 01:19:07.969819 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":389,"threads_started":1}
I20260806 01:19:07.972668 23245 heartbeater.cc:347] Connected to a master server at 127.25.236.190:33945
I20260806 01:19:07.972981 23245 heartbeater.cc:464] Registering TS with master...
I20260806 01:19:07.973862 23245 heartbeater.cc:511] Master 127.25.236.190:33945 requested a full tablet report, sending...
I20260806 01:19:07.975389 22772 ts_manager.cc:194] Registered new tserver with Master: e4e5016d49a94d69a5a3e565e95cd8dc (127.25.236.130:43277)
I20260806 01:19:07.976176 22772 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.236.130:36895
W20260806 01:19:08.322113 23249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:08.322486 23249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:08.322594 23249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:08.329618 23249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:08.329701 23249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:08.329805 23249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:19:08.338389 23249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:45349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:33945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:08.339615 23249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:08.341336 23249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:08.354156 23261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:08.354254 23258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:08.354204 23259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:08.354919 23249 server_base.cc:1034] running on GCE node
I20260806 01:19:08.355644 23249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:08.356588 23249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:08.357815 23249 hybrid_clock.cc:648] HybridClock initialized: now 1785979148357753 us; error 39 us; skew 500 ppm
I20260806 01:19:08.358144 23249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:19:08.360333 23249 webserver.cc:460] Webserver started at http://127.25.236.131:40017/ using document root <none> and password file <none>
I20260806 01:19:08.361006 23249 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:08.361104 23249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:08.361451 23249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:08.364390 23249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "23a81bbb8aae4204b7c407549cc44b42"
format_stamp: "Formatted at 2026-08-06 01:19:08 on dist-test-slave-7nm7"
server_key: "ac46c7adfa3dee7c0f6e1d0db73cd8d8"
server_key_iv: "fcc79aa2e5e7724ba79aefdd88427b37"
server_key_version: "encryptionkey@0"
I20260806 01:19:08.365191 23249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "23a81bbb8aae4204b7c407549cc44b42"
format_stamp: "Formatted at 2026-08-06 01:19:08 on dist-test-slave-7nm7"
server_key: "ac46c7adfa3dee7c0f6e1d0db73cd8d8"
server_key_iv: "fcc79aa2e5e7724ba79aefdd88427b37"
server_key_version: "encryptionkey@0"
I20260806 01:19:08.370007 23249 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260806 01:19:08.373420 23273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:08.374691 23249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:08.374862 23249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "23a81bbb8aae4204b7c407549cc44b42"
format_stamp: "Formatted at 2026-08-06 01:19:08 on dist-test-slave-7nm7"
server_key: "ac46c7adfa3dee7c0f6e1d0db73cd8d8"
server_key_iv: "fcc79aa2e5e7724ba79aefdd88427b37"
server_key_version: "encryptionkey@0"
I20260806 01:19:08.375059 23249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:08.389367 23249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:08.390179 23249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:08.390573 23249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:08.391557 23249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:19:08.392942 23249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:19:08.393060 23249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:08.393177 23249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:19:08.393302 23249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:08.428260 23249 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:46277
I20260806 01:19:08.428359 23437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:46277 every 8 connection(s)
I20260806 01:19:08.429790 23249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:19:08.438028 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 23249
I20260806 01:19:08.438311 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:19:08.438901 26546 external_mini_cluster.cc:1428] Setting key 866ced87d017c456254437279d16f2f2
I20260806 01:19:08.447754 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.432616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:33989 (local address 127.25.236.190:33945)
0806 01:19:08.432797 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:08.432802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:08.434314 (+ 1512us) server_negotiation.cc:407] Connection header received
0806 01:19:08.436059 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:08.436063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:08.436143 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:08.436236 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:08.437266 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.438016 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.440355 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.441209 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.441358 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.446844 (+ 5486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:08.446850 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:08.446861 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:08.446881 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:08.446916 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:08.446928 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:08.447029 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:08.447308 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 01:19:08.447411 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:19:08.448807 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.433160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:33945 (local address 127.25.236.131:33989)
0806 01:19:08.434042 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:08.434076 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:08.435809 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:08.436435 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:08.436443 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:08.436521 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:19:08.437093 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:08.437113 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.438182 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.438189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:08.440200 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:08.440219 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.441514 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.441523 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:08.441843 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.446456 (+ 4613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:19:08.446460 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:19:08.446516 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:19:08.446631 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:19:08.447084 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:19:08.447093 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:19:08.447099 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 01:19:08.447277 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 01:19:08.447688 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":273,"threads_started":1}
I20260806 01:19:08.450173 23438 heartbeater.cc:347] Connected to a master server at 127.25.236.190:33945
I20260806 01:19:08.450516 23438 heartbeater.cc:464] Registering TS with master...
I20260806 01:19:08.451222 23438 heartbeater.cc:511] Master 127.25.236.190:33945 requested a full tablet report, sending...
I20260806 01:19:08.452585 22772 ts_manager.cc:194] Registered new tserver with Master: 23a81bbb8aae4204b7c407549cc44b42 (127.25.236.131:46277)
I20260806 01:19:08.453421 22772 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.236.131:33989
I20260806 01:19:08.455456 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:19:08.465207 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.459460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46710 (local address 127.25.236.190:33945)
0806 01:19:08.459656 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:08.459660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:08.460177 (+ 517us) server_negotiation.cc:407] Connection header received
0806 01:19:08.460303 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:08.460307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:08.460366 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:08.460454 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:08.460890 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.461567 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.462982 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.463399 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.463493 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.464597 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:08.464601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:08.464605 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:08.464621 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:08.464650 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:08.464661 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:08.464772 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:08.464923 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 01:19:08.464964 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:19:08.468833 22772 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 01:19:08.474938 23043 heartbeater.cc:503] Master 127.25.236.190:33945 was elected leader, sending a full tablet report...
W20260806 01:19:08.476122 22772 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:19:08.500844 23458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.488503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49438 (local address 127.25.236.131:46277)
0806 01:19:08.489328 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:08.489337 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:08.490085 (+ 748us) server_negotiation.cc:407] Connection header received
0806 01:19:08.491483 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:08.491492 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:08.491772 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:08.491949 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:08.492972 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.494673 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.497821 (+ 3148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.499607 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.499809 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.500040 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:08.500312 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 01:19:08.500499 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":532,"threads_started":1}
I20260806 01:19:08.500941 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.489781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:43277 (local address 127.0.0.1:45262)
0806 01:19:08.491750 (+ 1969us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:08.491774 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:08.491915 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:08.493130 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:08.493136 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:08.493162 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:08.493457 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:08.493465 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.495396 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.495401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:08.497383 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:08.497442 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.500205 (+ 2763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.500215 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:08.500377 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.500410 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:19:08.500490 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 01:19:08.500610 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1800,"mutex_wait_us":4,"thread_start_us":721,"threads_started":1}
I20260806 01:19:08.501371 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.490352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45262 (local address 127.25.236.130:43277)
0806 01:19:08.492466 (+ 2114us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:08.492478 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:08.492526 (+ 48us) server_negotiation.cc:407] Connection header received
0806 01:19:08.492637 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:08.492646 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:08.492893 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:08.493608 (+ 715us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:08.493755 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.495243 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.497840 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.499611 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.499828 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.500039 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:08.500927 (+ 888us) server_negotiation.cc:299] Negotiation successful
0806 01:19:08.501111 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1905,"thread_start_us":1971,"threads_started":1}
I20260806 01:19:08.501439 23456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.488269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:46277 (local address 127.0.0.1:49438)
0806 01:19:08.489929 (+ 1660us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:08.489975 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:08.490288 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:08.492361 (+ 2073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:08.492368 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:08.492409 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:08.492781 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:08.492791 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.494843 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.494848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:08.497466 (+ 2618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:08.497485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.499826 (+ 2341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.499832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:08.500028 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.500084 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 01:19:08.500237 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 01:19:08.500423 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1391,"mutex_wait_us":73,"thread_start_us":1216,"threads_started":1}
I20260806 01:19:08.502257 23461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.488611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:38001 (local address 127.0.0.1:33572)
0806 01:19:08.490596 (+ 1985us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:08.490619 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:08.490754 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:08.493930 (+ 3176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:08.493934 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:08.493959 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:08.494362 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:08.494374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.496194 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.496199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:08.498651 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:08.498668 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.501470 (+ 2802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.501481 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:08.501684 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.501722 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 01:19:08.501807 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 01:19:08.501933 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1809,"thread_start_us":2771,"threads_started":1}
I20260806 01:19:08.502485 23460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.489329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33572 (local address 127.25.236.129:38001)
0806 01:19:08.493384 (+ 4055us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:08.493396 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:08.493422 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:19:08.493515 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:08.493522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:08.493731 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:08.493911 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:08.494558 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.496034 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.498906 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.500342 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.500533 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.500727 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:08.501891 (+ 1164us) server_negotiation.cc:299] Negotiation successful
0806 01:19:08.502058 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3877,"thread_start_us":3719,"threads_started":1}
I20260806 01:19:08.505641 22949 tablet_service.cc:1467] Processing CreateTablet for tablet d7e2044a77ab449c8d9d279abbc7af60 (DEFAULT_TABLE table=test-table [id=b166068a59964bd9bfb34b2a61e76aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:08.505477 23340 tablet_service.cc:1467] Processing CreateTablet for tablet d7e2044a77ab449c8d9d279abbc7af60 (DEFAULT_TABLE table=test-table [id=b166068a59964bd9bfb34b2a61e76aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:08.505477 23150 tablet_service.cc:1467] Processing CreateTablet for tablet d7e2044a77ab449c8d9d279abbc7af60 (DEFAULT_TABLE table=test-table [id=b166068a59964bd9bfb34b2a61e76aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:08.506916 22949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7e2044a77ab449c8d9d279abbc7af60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:08.506935 23150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7e2044a77ab449c8d9d279abbc7af60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:08.506935 23340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7e2044a77ab449c8d9d279abbc7af60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:08.518091 23463 tablet_bootstrap.cc:492] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f: Bootstrap starting.
I20260806 01:19:08.521093 23464 tablet_bootstrap.cc:492] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc: Bootstrap starting.
I20260806 01:19:08.521929 23463 tablet_bootstrap.cc:654] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:08.523337 23463 log.cc:824] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:08.523886 23465 tablet_bootstrap.cc:492] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42: Bootstrap starting.
I20260806 01:19:08.525076 23464 tablet_bootstrap.cc:654] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:08.526500 23464 log.cc:824] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:08.526823 23465 tablet_bootstrap.cc:654] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:08.528097 23465 log.cc:824] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:08.529047 23463 tablet_bootstrap.cc:492] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f: No bootstrap required, opened a new log
I20260806 01:19:08.529099 23464 tablet_bootstrap.cc:492] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc: No bootstrap required, opened a new log
I20260806 01:19:08.529356 23464 ts_tablet_manager.cc:1397] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260806 01:19:08.529347 23463 ts_tablet_manager.cc:1397] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260806 01:19:08.531198 23465 tablet_bootstrap.cc:492] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42: No bootstrap required, opened a new log
I20260806 01:19:08.531478 23465 ts_tablet_manager.cc:1397] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 01:19:08.536798 23464 raft_consensus.cc:348] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a81bbb8aae4204b7c407549cc44b42" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46277 } } peers { permanent_uuid: "521bf1ef556a4abb92a82a112abdcd0f" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38001 } } peers { permanent_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43277 } }
I20260806 01:19:08.537139 23464 raft_consensus.cc:374] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:08.537245 23464 raft_consensus.cc:729] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4e5016d49a94d69a5a3e565e95cd8dc, State: Initialized, Role: FOLLOWER
I20260806 01:19:08.537854 23464 consensus_queue.cc:260] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a81bbb8aae4204b7c407549cc44b42" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46277 } } peers { permanent_uuid: "521bf1ef556a4abb92a82a112abdcd0f" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38001 } } peers { permanent_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43277 } }
I20260806 01:19:08.538947 23245 heartbeater.cc:503] Master 127.25.236.190:33945 was elected leader, sending a full tablet report...
I20260806 01:19:08.539567 23464 ts_tablet_manager.cc:1428] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc: Time spent starting tablet: real 0.010s user 0.007s sys 0.002s
I20260806 01:19:08.540997 23463 raft_consensus.cc:348] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a81bbb8aae4204b7c407549cc44b42" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46277 } } peers { permanent_uuid: "521bf1ef556a4abb92a82a112abdcd0f" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38001 } } peers { permanent_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43277 } }
I20260806 01:19:08.541422 23463 raft_consensus.cc:374] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:08.541561 23463 raft_consensus.cc:729] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 521bf1ef556a4abb92a82a112abdcd0f, State: Initialized, Role: FOLLOWER
I20260806 01:19:08.541759 23465 raft_consensus.cc:348] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a81bbb8aae4204b7c407549cc44b42" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46277 } } peers { permanent_uuid: "521bf1ef556a4abb92a82a112abdcd0f" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38001 } } peers { permanent_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43277 } }
I20260806 01:19:08.542181 23465 raft_consensus.cc:374] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:08.542342 23465 raft_consensus.cc:729] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23a81bbb8aae4204b7c407549cc44b42, State: Initialized, Role: FOLLOWER
I20260806 01:19:08.542220 23463 consensus_queue.cc:260] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a81bbb8aae4204b7c407549cc44b42" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46277 } } peers { permanent_uuid: "521bf1ef556a4abb92a82a112abdcd0f" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38001 } } peers { permanent_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43277 } }
I20260806 01:19:08.543113 23465 consensus_queue.cc:260] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a81bbb8aae4204b7c407549cc44b42" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46277 } } peers { permanent_uuid: "521bf1ef556a4abb92a82a112abdcd0f" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38001 } } peers { permanent_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43277 } }
I20260806 01:19:08.544245 23463 ts_tablet_manager.cc:1428] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260806 01:19:08.545652 23438 heartbeater.cc:503] Master 127.25.236.190:33945 was elected leader, sending a full tablet report...
I20260806 01:19:08.546208 23465 ts_tablet_manager.cc:1428] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
W20260806 01:19:08.681671 23439 tablet.cc:2369] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:19:08.700757 23044 tablet.cc:2369] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:19:08.706533 23247 tablet.cc:2369] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:19:08.758909 23473 raft_consensus.cc:482] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:19:08.759243 23473 raft_consensus.cc:504] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a81bbb8aae4204b7c407549cc44b42" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46277 } } peers { permanent_uuid: "521bf1ef556a4abb92a82a112abdcd0f" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38001 } } peers { permanent_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43277 } }
I20260806 01:19:08.761029 23473 leader_election.cc:288] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23a81bbb8aae4204b7c407549cc44b42 (127.25.236.131:46277), 521bf1ef556a4abb92a82a112abdcd0f (127.25.236.129:38001)
I20260806 01:19:08.767724 23460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.761543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:43325 (local address 127.25.236.129:38001)
0806 01:19:08.761734 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:08.761741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:08.762394 (+ 653us) server_negotiation.cc:407] Connection header received
0806 01:19:08.762626 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:08.762630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:08.762701 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:08.762799 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:08.763524 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.764214 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.765987 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.766820 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.767014 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.767087 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:08.767357 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 01:19:08.767480 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 01:19:08.767882 23476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.761333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:38001 (local address 127.25.236.130:43325)
0806 01:19:08.762223 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:08.762287 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:08.762477 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:08.762828 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:08.762832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:08.762856 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:08.763377 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:08.763388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.764346 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.764354 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:08.765788 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:08.765801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.767018 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.767024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:08.767195 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.767219 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:19:08.767329 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 01:19:08.767456 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":4,"thread_start_us":545,"threads_started":1}
I20260806 01:19:08.768474 23458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.761394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:40847 (local address 127.25.236.131:46277)
0806 01:19:08.761638 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:08.761647 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:08.762480 (+ 833us) server_negotiation.cc:407] Connection header received
0806 01:19:08.762683 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:08.762690 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:08.762755 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:08.762852 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:08.763986 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.764934 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.766700 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.767445 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.767637 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.767704 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:08.768118 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 01:19:08.768240 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 01:19:08.768522 23477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.761418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:46277 (local address 127.25.236.130:40847)
0806 01:19:08.762373 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:08.762388 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:08.762542 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:08.762867 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:08.762871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:08.762894 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:08.763628 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:08.763647 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.765098 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.765106 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:08.766518 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:08.766531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.767656 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.767662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:08.767893 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.767925 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:19:08.768048 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 01:19:08.768181 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":44,"thread_start_us":596,"threads_started":1}
I20260806 01:19:08.768999 22981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7e2044a77ab449c8d9d279abbc7af60" candidate_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "521bf1ef556a4abb92a82a112abdcd0f" is_pre_election: true
I20260806 01:19:08.769472 23369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7e2044a77ab449c8d9d279abbc7af60" candidate_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23a81bbb8aae4204b7c407549cc44b42" is_pre_election: true
I20260806 01:19:08.769635 22981 raft_consensus.cc:2454] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4e5016d49a94d69a5a3e565e95cd8dc in term 0.
I20260806 01:19:08.769994 23369 raft_consensus.cc:2454] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4e5016d49a94d69a5a3e565e95cd8dc in term 0.
I20260806 01:19:08.770390 23072 leader_election.cc:302] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 521bf1ef556a4abb92a82a112abdcd0f, e4e5016d49a94d69a5a3e565e95cd8dc; no voters:
I20260806 01:19:08.770792 23473 raft_consensus.cc:2781] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:19:08.770937 23473 raft_consensus.cc:482] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:19:08.771047 23473 raft_consensus.cc:3037] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:08.773496 23473 raft_consensus.cc:504] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a81bbb8aae4204b7c407549cc44b42" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46277 } } peers { permanent_uuid: "521bf1ef556a4abb92a82a112abdcd0f" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38001 } } peers { permanent_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43277 } }
I20260806 01:19:08.774196 23473 leader_election.cc:288] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [CANDIDATE]: Term 1 election: Requested vote from peers 23a81bbb8aae4204b7c407549cc44b42 (127.25.236.131:46277), 521bf1ef556a4abb92a82a112abdcd0f (127.25.236.129:38001)
I20260806 01:19:08.774806 23369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7e2044a77ab449c8d9d279abbc7af60" candidate_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23a81bbb8aae4204b7c407549cc44b42"
I20260806 01:19:08.774813 22981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7e2044a77ab449c8d9d279abbc7af60" candidate_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "521bf1ef556a4abb92a82a112abdcd0f"
I20260806 01:19:08.775074 22981 raft_consensus.cc:3037] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:08.775065 23369 raft_consensus.cc:3037] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:08.777721 23369 raft_consensus.cc:2454] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4e5016d49a94d69a5a3e565e95cd8dc in term 1.
I20260806 01:19:08.777962 22981 raft_consensus.cc:2454] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4e5016d49a94d69a5a3e565e95cd8dc in term 1.
I20260806 01:19:08.778298 23076 leader_election.cc:302] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23a81bbb8aae4204b7c407549cc44b42, e4e5016d49a94d69a5a3e565e95cd8dc; no voters:
I20260806 01:19:08.778646 23473 raft_consensus.cc:2781] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:08.779201 23473 raft_consensus.cc:686] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [term 1 LEADER]: Becoming Leader. State: Replica: e4e5016d49a94d69a5a3e565e95cd8dc, State: Running, Role: LEADER
I20260806 01:19:08.779844 23473 consensus_queue.cc:237] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a81bbb8aae4204b7c407549cc44b42" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46277 } } peers { permanent_uuid: "521bf1ef556a4abb92a82a112abdcd0f" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38001 } } peers { permanent_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43277 } }
I20260806 01:19:08.785346 22772 catalog_manager.cc:5495] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc reported cstate change: term changed from 0 to 1, leader changed from <none> to e4e5016d49a94d69a5a3e565e95cd8dc (127.25.236.130). New cstate: current_term: 1 leader_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23a81bbb8aae4204b7c407549cc44b42" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "521bf1ef556a4abb92a82a112abdcd0f" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4e5016d49a94d69a5a3e565e95cd8dc" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 43277 } health_report { overall_health: HEALTHY } } }
I20260806 01:19:08.873699 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.867816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45272 (local address 127.25.236.130:43277)
0806 01:19:08.868033 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:08.868038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:08.868057 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:19:08.868270 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:08.868275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:08.868366 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:08.868525 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:19:08.868901 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.869590 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.871048 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.871460 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.871598 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.872217 (+ 619us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:19:08.872952 (+ 735us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:19:08.873186 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:08.873404 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 01:19:08.873457 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:19:08.881906 22981 raft_consensus.cc:1261] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f [term 1 FOLLOWER]: Refusing update from remote peer e4e5016d49a94d69a5a3e565e95cd8dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:19:08.881907 23369 raft_consensus.cc:1261] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42 [term 1 FOLLOWER]: Refusing update from remote peer e4e5016d49a94d69a5a3e565e95cd8dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:19:08.883195 23478 consensus_queue.cc:1035] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "23a81bbb8aae4204b7c407549cc44b42" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:19:08.883653 23473 consensus_queue.cc:1035] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "521bf1ef556a4abb92a82a112abdcd0f" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 38001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:19:08.895634 23483 mvcc.cc:205] Tried to move back new op lower bound from 7315370593810522112 to 7315370593406140416. Current Snapshot: MvccSnapshot[applied={T|T < 7315370593810522112}]
I20260806 01:19:08.897245 23482 mvcc.cc:205] Tried to move back new op lower bound from 7315370593810522112 to 7315370593406140416. Current Snapshot: MvccSnapshot[applied={T|T < 7315370593810522112}]
I20260806 01:19:08.911971 22770 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:19:08.912385 22770 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:19:08.916426 22770 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65fae9f02af24a1ea512ba91f55998cb: Sending DeleteTablet for 3 replicas of tablet d7e2044a77ab449c8d9d279abbc7af60
I20260806 01:19:08.917737 23150 tablet_service.cc:1514] Processing DeleteTablet for tablet d7e2044a77ab449c8d9d279abbc7af60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:08 UTC) from {username='slave'} at 127.0.0.1:45262
I20260806 01:19:08.917763 22949 tablet_service.cc:1514] Processing DeleteTablet for tablet d7e2044a77ab449c8d9d279abbc7af60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:08 UTC) from {username='slave'} at 127.0.0.1:33572
I20260806 01:19:08.917763 23340 tablet_service.cc:1514] Processing DeleteTablet for tablet d7e2044a77ab449c8d9d279abbc7af60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:08 UTC) from {username='slave'} at 127.0.0.1:49438
I20260806 01:19:08.920252 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 22862
I20260806 01:19:08.931380 23495 tablet_replica.cc:330] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f: stopping tablet replica
I20260806 01:19:08.932138 23495 raft_consensus.cc:2229] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:19:08.932868 23495 raft_consensus.cc:2258] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:08.936738 23495 ts_tablet_manager.cc:1905] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:08.937469 23493 tablet_replica.cc:330] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42: stopping tablet replica
I20260806 01:19:08.938628 23496 tablet_replica.cc:330] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc: stopping tablet replica
I20260806 01:19:08.939445 23496 raft_consensus.cc:2229] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:19:08.940189 23496 raft_consensus.cc:2258] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:08.942757 23496 ts_tablet_manager.cc:1905] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:08.946542 23460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:08.921708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33578 (local address 127.25.236.129:38001)
0806 01:19:08.921941 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:08.921951 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:08.937741 (+ 15790us) server_negotiation.cc:407] Connection header received
0806 01:19:08.937949 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:08.937957 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:08.938076 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:08.938186 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:08.938714 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.940115 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.942342 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:08.943036 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:08.943145 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:08.944728 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:08.944759 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:08.944777 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:08.945585 (+ 808us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:08.945650 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:08.945679 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:08.945769 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:08.946003 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 01:19:08.946127 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 01:19:08.947009 23495 ts_tablet_manager.cc:1918] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:08.947250 23495 log.cc:1196] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/d7e2044a77ab449c8d9d279abbc7af60
I20260806 01:19:08.947916 23495 ts_tablet_manager.cc:1939] T d7e2044a77ab449c8d9d279abbc7af60 P 521bf1ef556a4abb92a82a112abdcd0f: Deleting consensus metadata
I20260806 01:19:08.948100 23033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:08.949767 23493 raft_consensus.cc:2229] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:19:08.950775 23493 raft_consensus.cc:2258] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:08.951059 23496 ts_tablet_manager.cc:1918] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:08.951355 23496 log.cc:1196] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/d7e2044a77ab449c8d9d279abbc7af60
I20260806 01:19:08.952656 23496 ts_tablet_manager.cc:1939] T d7e2044a77ab449c8d9d279abbc7af60 P e4e5016d49a94d69a5a3e565e95cd8dc: Deleting consensus metadata
I20260806 01:19:08.955539 23493 ts_tablet_manager.cc:1905] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:08.961400 22749 catalog_manager.cc:4841] TS e4e5016d49a94d69a5a3e565e95cd8dc (127.25.236.130:43277): tablet d7e2044a77ab449c8d9d279abbc7af60 (table test-table [id=b166068a59964bd9bfb34b2a61e76aba]) successfully deleted
I20260806 01:19:08.970321 23493 ts_tablet_manager.cc:1918] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:08.970664 23493 log.cc:1196] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/d7e2044a77ab449c8d9d279abbc7af60
I20260806 01:19:08.971283 23493 ts_tablet_manager.cc:1939] T d7e2044a77ab449c8d9d279abbc7af60 P 23a81bbb8aae4204b7c407549cc44b42: Deleting consensus metadata
I20260806 01:19:08.973450 22754 catalog_manager.cc:4841] TS 23a81bbb8aae4204b7c407549cc44b42 (127.25.236.131:46277): tablet d7e2044a77ab449c8d9d279abbc7af60 (table test-table [id=b166068a59964bd9bfb34b2a61e76aba]) successfully deleted
I20260806 01:19:09.063560 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 22862
W20260806 01:19:09.094802 22751 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.236.129:38001 (error 108)
I20260806 01:19:09.095955 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 23048
I20260806 01:19:09.097577 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:09.096255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:38001 (local address 127.0.0.1:33588)
0806 01:19:09.097105 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:09.097251 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":806,"threads_started":1}
W20260806 01:19:09.097965 22751 catalog_manager.cc:4584] TS 521bf1ef556a4abb92a82a112abdcd0f (127.25.236.129:38001): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d7e2044a77ab449c8d9d279abbc7af60: Network error: Client connection negotiation failed: client connection to 127.25.236.129:38001: connect: Connection refused (error 111)
I20260806 01:19:09.105001 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:09.096850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.25.236.130:43277)
0806 01:19:09.097048 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:09.097054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:09.097074 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:19:09.097151 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:09.097156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:09.097269 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:09.097532 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:09.098010 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:09.099460 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:09.101345 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:09.101800 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:09.101897 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:09.103459 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:09.103471 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:09.103485 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:09.104193 (+ 708us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:09.104265 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:09.104294 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:09.104400 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:09.104555 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 01:19:09.104636 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 01:19:09.105680 23235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:09.151669 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:09.151202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:38001 (local address 127.0.0.1:33602)
0806 01:19:09.151427 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:09.151514 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 01:19:09.152676 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:09.152313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:38001 (local address 127.0.0.1:33604)
0806 01:19:09.152480 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:09.152546 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 01:19:09.213296 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:09.212765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:38001 (local address 127.0.0.1:33612)
0806 01:19:09.213032 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:09.213139 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 01:19:09.214391 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:09.213963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:38001 (local address 127.0.0.1:33618)
0806 01:19:09.214114 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:09.214190 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 01:19:09.218954 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 23048
I20260806 01:19:09.244511 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 23249
I20260806 01:19:09.251387 23458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:09.245341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49454 (local address 127.25.236.131:46277)
0806 01:19:09.245533 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:09.245541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:09.245619 (+ 78us) server_negotiation.cc:407] Connection header received
0806 01:19:09.245858 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:09.245862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:09.245942 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:09.246056 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:09.246488 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:09.247212 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:09.248458 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:09.248867 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:09.248999 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:09.250011 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:09.250020 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:09.250034 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:09.250659 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:09.250703 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:09.250723 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:09.250847 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:09.250952 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 01:19:09.251056 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 01:19:09.252099 23433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:09.295117 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:09.294624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:38001 (local address 127.0.0.1:33630)
0806 01:19:09.294881 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:09.294967 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 01:19:09.296172 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:09.295767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:38001 (local address 127.0.0.1:33640)
0806 01:19:09.295970 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:09.296045 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.129:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260806 01:19:09.342664 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 23249
I20260806 01:19:09.367668 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 22731
I20260806 01:19:09.368768 22817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:09.478628 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 22731
2026-08-06T01:19:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3181 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:09 dist-test-slave-7nm7 krb5kdc[23511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:09 dist-test-slave-7nm7 krb5kdc[23511](info): set up 2 sockets
Aug 06 01:19:09 dist-test-slave-7nm7 krb5kdc[23511](info): commencing operation
krb5kdc: starting...
W20260806 01:19:11.160151 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.632s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:11 dist-test-slave-7nm7 krb5kdc[23511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
W20260806 01:19:11.730458 26546 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.25.236.190: real 0.126s user 0.003s sys 0.048s
I20260806 01:19:11.732359 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:44573
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:44573
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:12.612399 23530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:12.613024 23530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:12.613353 23530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:12.623193 23530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:12.623319 23530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:12.623407 23530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:12.623495 23530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:12.623611 23530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:12.634892 23530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:44573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:44573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:12.636504 23530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:12.638072 23530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:12.651897 23538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:12.652124 23535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:12.651898 23545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:12.653187 23530 server_base.cc:1034] running on GCE node
I20260806 01:19:12.654066 23530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:12.655682 23530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:12.656910 23530 hybrid_clock.cc:648] HybridClock initialized: now 1785979152656841 us; error 53 us; skew 500 ppm
I20260806 01:19:12.657235 23530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:12 dist-test-slave-7nm7 krb5kdc[23511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979152, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:12.660763 23530 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:19:12.662377 23530 webserver.cc:460] Webserver started at http://127.25.236.190:45173/ using document root <none> and password file <none>
I20260806 01:19:12.663028 23530 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:12.663139 23530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:12.663448 23530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:12.666581 23530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "0a3aa09e108540c3b6d870b04bfbb7aa"
format_stamp: "Formatted at 2026-08-06 01:19:12 on dist-test-slave-7nm7"
server_key: "d2a2389aa0f622446b2ec58b407d66bf"
server_key_iv: "d830985d6938c93e32f81e8a10104c3a"
server_key_version: "encryptionkey@0"
I20260806 01:19:12.667395 23530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "0a3aa09e108540c3b6d870b04bfbb7aa"
format_stamp: "Formatted at 2026-08-06 01:19:12 on dist-test-slave-7nm7"
server_key: "d2a2389aa0f622446b2ec58b407d66bf"
server_key_iv: "d830985d6938c93e32f81e8a10104c3a"
server_key_version: "encryptionkey@0"
I20260806 01:19:12.673054 23530 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 01:19:12.676841 23556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:12.678406 23530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:19:12.678593 23530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "0a3aa09e108540c3b6d870b04bfbb7aa"
format_stamp: "Formatted at 2026-08-06 01:19:12 on dist-test-slave-7nm7"
server_key: "d2a2389aa0f622446b2ec58b407d66bf"
server_key_iv: "d830985d6938c93e32f81e8a10104c3a"
server_key_version: "encryptionkey@0"
I20260806 01:19:12.678778 23530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:12.698871 23530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:12.703729 23530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:12.704018 23530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:12.725862 23530 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:44573
I20260806 01:19:12.725893 23625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:44573 every 8 connection(s)
I20260806 01:19:12.727378 23530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:12.731600 23627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:12.733508 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 23530
I20260806 01:19:12.733770 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:12.734404 26546 external_mini_cluster.cc:1428] Setting key f88812b08adc086e4104efa16a574c95
I20260806 01:19:12.746280 23627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa: Bootstrap starting.
Aug 06 01:19:12 dist-test-slave-7nm7 krb5kdc[23511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:19:12.750300 23627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:12.751435 23627 log.cc:824] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:12.754571 23627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa: No bootstrap required, opened a new log
I20260806 01:19:12.758224 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:12.736640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37004 (local address 127.25.236.190:44573)
0806 01:19:12.737533 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:12.737544 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:12.737593 (+ 49us) server_negotiation.cc:407] Connection header received
0806 01:19:12.739351 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:12.739374 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:12.739755 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:12.740161 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:12.741793 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:12.743165 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:12.744489 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:12.745160 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:12.745369 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:12.749931 (+ 4562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:12.749940 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:12.749954 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:12.749997 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:12.753741 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:12.754586 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:12.754594 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:12.754600 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:12.754823 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:12.755230 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:12.755235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:12.755238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:12.755792 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:12.756104 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:12.756601 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 01:19:12.757049 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":383,"threads_started":1}
I20260806 01:19:12.765743 23627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a3aa09e108540c3b6d870b04bfbb7aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 44573 } }
I20260806 01:19:12.766158 23627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:12.766302 23627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3aa09e108540c3b6d870b04bfbb7aa, State: Initialized, Role: FOLLOWER
I20260806 01:19:12.767005 23627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a3aa09e108540c3b6d870b04bfbb7aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 44573 } }
I20260806 01:19:12.767275 23627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:12.767437 23627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:12.767618 23627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:12.770388 23627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a3aa09e108540c3b6d870b04bfbb7aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 44573 } }
I20260806 01:19:12.770964 23627 leader_election.cc:302] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a3aa09e108540c3b6d870b04bfbb7aa; no voters:
I20260806 01:19:12.771586 23627 leader_election.cc:288] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:12.771781 23637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:12.772848 23637 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [term 1 LEADER]: Becoming Leader. State: Replica: 0a3aa09e108540c3b6d870b04bfbb7aa, State: Running, Role: LEADER
I20260806 01:19:12.773413 23637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a3aa09e108540c3b6d870b04bfbb7aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 44573 } }
I20260806 01:19:12.773924 23627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:12.777175 23638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a3aa09e108540c3b6d870b04bfbb7aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a3aa09e108540c3b6d870b04bfbb7aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 44573 } } }
I20260806 01:19:12.777587 23638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:12.777102 23639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a3aa09e108540c3b6d870b04bfbb7aa. Latest consensus state: current_term: 1 leader_uuid: "0a3aa09e108540c3b6d870b04bfbb7aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a3aa09e108540c3b6d870b04bfbb7aa" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 44573 } } }
I20260806 01:19:12.778267 23639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:12.779887 23646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:12.788100 23646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:12.799703 23646 catalog_manager.cc:1347] Generated new cluster ID: 792b59174c3e43c3b6111b3f25aa88ed
I20260806 01:19:12.799841 23646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:12.808184 23646 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:12.810077 23646 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:12.829339 23646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa: Generated new TSK 0
I20260806 01:19:12.830371 23646 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
Entry for principal kudu/127.25.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab.
I20260806 01:19:12.941625 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:44573
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:13.346982 23663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:13.347414 23663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:13.347606 23663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:13.357101 23663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:13.357295 23663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:13.357498 23663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:19:13.368182 23663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.129.keytab
--principal=kudu/127.25.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:44573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:13.369618 23663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:13.371016 23663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:13.385425 23672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:13.385351 23675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:13.385372 23673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:13.387008 23663 server_base.cc:1034] running on GCE node
I20260806 01:19:13.387916 23663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:13.388974 23663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:13.390358 23663 hybrid_clock.cc:648] HybridClock initialized: now 1785979153390314 us; error 61 us; skew 500 ppm
I20260806 01:19:13.390715 23663 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:13 dist-test-slave-7nm7 krb5kdc[23511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979153, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:13.394760 23663 init.cc:377] Logged in from keytab as kudu/127.25.236.129@KRBTEST.COM (short username kudu)
I20260806 01:19:13.396673 23663 webserver.cc:460] Webserver started at http://127.25.236.129:45921/ using document root <none> and password file <none>
I20260806 01:19:13.397605 23663 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:13.397779 23663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:13.398345 23663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:13.401707 23663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "c335dbd912594634822e8db891b48fdd"
format_stamp: "Formatted at 2026-08-06 01:19:13 on dist-test-slave-7nm7"
server_key: "d82ce41295947ec2090bdafac5526c37"
server_key_iv: "61a1430635259192913c296ca48921de"
server_key_version: "encryptionkey@0"
I20260806 01:19:13.402568 23663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "c335dbd912594634822e8db891b48fdd"
format_stamp: "Formatted at 2026-08-06 01:19:13 on dist-test-slave-7nm7"
server_key: "d82ce41295947ec2090bdafac5526c37"
server_key_iv: "61a1430635259192913c296ca48921de"
server_key_version: "encryptionkey@0"
I20260806 01:19:13.408030 23663 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:19:13.411652 23686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:13.413046 23663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:19:13.413349 23663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "c335dbd912594634822e8db891b48fdd"
format_stamp: "Formatted at 2026-08-06 01:19:13 on dist-test-slave-7nm7"
server_key: "d82ce41295947ec2090bdafac5526c37"
server_key_iv: "61a1430635259192913c296ca48921de"
server_key_version: "encryptionkey@0"
I20260806 01:19:13.413568 23663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:13.438930 23663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:13.443843 23663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:13.444167 23663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:13.445266 23663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:19:13.446787 23663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:19:13.446892 23663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:13.447041 23663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:19:13.447116 23663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:13.487252 23663 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:33105
I20260806 01:19:13.487344 23842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:33105 every 8 connection(s)
I20260806 01:19:13.488521 23663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:19:13.490001 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 23663
I20260806 01:19:13.490211 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:19:13.490823 26546 external_mini_cluster.cc:1428] Setting key f206ce38bfbe54e82321f0d0ef78461d
Aug 06 01:19:13 dist-test-slave-7nm7 krb5kdc[23511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979153, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.129@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:19:13.516438 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:13.491483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:53339 (local address 127.25.236.190:44573)
0806 01:19:13.492020 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:13.492027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:13.493118 (+ 1091us) server_negotiation.cc:407] Connection header received
0806 01:19:13.495094 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:13.495100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:13.495181 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:13.495309 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:13.496795 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:13.497528 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:13.499149 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:13.499574 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:13.499713 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:13.508871 (+ 9158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:13.508877 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:13.508881 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:13.508918 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:13.511897 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:13.513183 (+ 1286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:13.513190 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:13.513193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:13.513348 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:13.513999 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:13.514003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:13.514005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:13.514399 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:13.514523 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:13.515939 (+ 1416us) server_negotiation.cc:299] Negotiation successful
0806 01:19:13.516144 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":289,"threads_started":1}
I20260806 01:19:13.519368 23851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:13.492054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:44573 (local address 127.25.236.129:53339)
0806 01:19:13.492958 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:13.492994 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:13.494828 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:13.495527 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:13.495535 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:13.496072 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:19:13.496629 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:13.496646 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:13.497708 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:13.497712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:13.498993 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:13.499005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:13.499819 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:13.499827 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:13.500062 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:13.505413 (+ 5351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:19:13.505438 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:19:13.508530 (+ 3092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:19:13.512081 (+ 3551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:19:13.512095 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:19:13.512116 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:19:13.512917 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:19:13.513510 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:19:13.513519 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:19:13.513526 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:19:13.513856 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:19:13.514616 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:19:13.514632 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:19:13.515345 (+ 713us) client_negotiation.cc:769] Sending connection context
0806 01:19:13.515815 (+ 470us) client_negotiation.cc:240] Negotiation successful
0806 01:19:13.517063 (+ 1248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":405,"threads_started":1}
I20260806 01:19:13.521477 23843 heartbeater.cc:347] Connected to a master server at 127.25.236.190:44573
I20260806 01:19:13.522038 23843 heartbeater.cc:464] Registering TS with master...
I20260806 01:19:13.523296 23843 heartbeater.cc:511] Master 127.25.236.190:44573 requested a full tablet report, sending...
I20260806 01:19:13.525974 23584 ts_manager.cc:194] Registered new tserver with Master: c335dbd912594634822e8db891b48fdd (127.25.236.129:33105)
I20260806 01:19:13.527503 23584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.129@KRBTEST.COM'} at 127.25.236.129:53339
WARNING: no policy specified for kudu/127.25.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
Entry for principal kudu/127.25.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab.
I20260806 01:19:13.576246 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:44573
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:13.954933 23855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:13.955556 23855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:13.955834 23855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:13.963862 23855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:13.963990 23855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:13.964125 23855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:19:13.974943 23855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.130.keytab
--principal=kudu/127.25.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:44573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:13.976411 23855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:13.977762 23855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:13.992080 23863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:13.992261 23864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:13.992132 23867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:13.993703 23855 server_base.cc:1034] running on GCE node
I20260806 01:19:13.994462 23855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:13.995476 23855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:13.996695 23855 hybrid_clock.cc:648] HybridClock initialized: now 1785979153996621 us; error 61 us; skew 500 ppm
I20260806 01:19:13.997035 23855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:13 dist-test-slave-7nm7 krb5kdc[23511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979153, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:14.000883 23855 init.cc:377] Logged in from keytab as kudu/127.25.236.130@KRBTEST.COM (short username kudu)
I20260806 01:19:14.002426 23855 webserver.cc:460] Webserver started at http://127.25.236.130:37539/ using document root <none> and password file <none>
I20260806 01:19:14.003167 23855 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:14.003322 23855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:14.003716 23855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:14.006819 23855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "a7fef7fa825b4600a4d6a34418fbf6c7"
format_stamp: "Formatted at 2026-08-06 01:19:13 on dist-test-slave-7nm7"
server_key: "1d33e28fe6944bbd44466c5ef37cdf30"
server_key_iv: "b4e128436e5fae016c4752433865334f"
server_key_version: "encryptionkey@0"
I20260806 01:19:14.007612 23855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "a7fef7fa825b4600a4d6a34418fbf6c7"
format_stamp: "Formatted at 2026-08-06 01:19:13 on dist-test-slave-7nm7"
server_key: "1d33e28fe6944bbd44466c5ef37cdf30"
server_key_iv: "b4e128436e5fae016c4752433865334f"
server_key_version: "encryptionkey@0"
I20260806 01:19:14.013500 23855 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:19:14.017534 23881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:14.019083 23855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:14.019279 23855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "a7fef7fa825b4600a4d6a34418fbf6c7"
format_stamp: "Formatted at 2026-08-06 01:19:13 on dist-test-slave-7nm7"
server_key: "1d33e28fe6944bbd44466c5ef37cdf30"
server_key_iv: "b4e128436e5fae016c4752433865334f"
server_key_version: "encryptionkey@0"
I20260806 01:19:14.019559 23855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:14.036919 23855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:14.040930 23855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:14.041337 23855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:14.042858 23855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:19:14.044909 23855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:19:14.045024 23855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:14.045176 23855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:19:14.045256 23855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:14.097213 23855 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:38267
I20260806 01:19:14.097266 24044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:38267 every 8 connection(s)
I20260806 01:19:14.099285 23855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:19:14.107646 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 23855
I20260806 01:19:14.107888 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:19:14.108708 26546 external_mini_cluster.cc:1428] Setting key 3719c8a5ccbe61976e6c4674d956f51a
Aug 06 01:19:14 dist-test-slave-7nm7 krb5kdc[23511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979153, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.130@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:19:14.129460 24048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:14.103435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:55375 (local address 127.25.236.190:44573)
0806 01:19:14.103995 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:14.104005 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:14.105224 (+ 1219us) server_negotiation.cc:407] Connection header received
0806 01:19:14.108323 (+ 3099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:14.108331 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:14.108436 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:14.108533 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:14.111220 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.112608 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.115325 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.116117 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.116270 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:14.121223 (+ 4953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:14.121232 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:14.121239 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:14.121287 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:14.125784 (+ 4497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:14.126722 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:14.126730 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:14.126736 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:14.126888 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:14.127418 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:14.127422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:14.127424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:14.127715 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:14.127826 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:14.128701 (+ 875us) server_negotiation.cc:299] Negotiation successful
0806 01:19:14.128954 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":273,"threads_started":1}
I20260806 01:19:14.130046 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:14.104007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:44573 (local address 127.25.236.130:55375)
0806 01:19:14.104912 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:14.104964 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:14.107958 (+ 2994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:14.109139 (+ 1181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:14.109153 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:14.110074 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:19:14.111008 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:14.111039 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.112782 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.112790 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:14.115169 (+ 2379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:14.115189 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.116642 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.116661 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:14.117062 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:14.118087 (+ 1025us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:19:14.118122 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:19:14.120962 (+ 2840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:19:14.125944 (+ 4982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:19:14.125951 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:19:14.125965 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:19:14.126569 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:19:14.127005 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:19:14.127011 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:19:14.127016 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:19:14.127280 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:19:14.127858 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:19:14.127870 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:19:14.128288 (+ 418us) client_negotiation.cc:769] Sending connection context
0806 01:19:14.128555 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 01:19:14.128948 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":298,"threads_started":1}
I20260806 01:19:14.131606 24045 heartbeater.cc:347] Connected to a master server at 127.25.236.190:44573
I20260806 01:19:14.131932 24045 heartbeater.cc:464] Registering TS with master...
I20260806 01:19:14.132856 24045 heartbeater.cc:511] Master 127.25.236.190:44573 requested a full tablet report, sending...
I20260806 01:19:14.134582 23584 ts_manager.cc:194] Registered new tserver with Master: a7fef7fa825b4600a4d6a34418fbf6c7 (127.25.236.130:38267)
I20260806 01:19:14.135427 23584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.130@KRBTEST.COM'} at 127.25.236.130:55375
WARNING: no policy specified for kudu/127.25.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
Entry for principal kudu/127.25.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab.
I20260806 01:19:14.185072 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:44573
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:19:14.531351 23843 heartbeater.cc:503] Master 127.25.236.190:44573 was elected leader, sending a full tablet report...
W20260806 01:19:14.580296 24052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:14.580598 24052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:14.580699 24052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:14.589077 24052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:14.589205 24052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:14.589329 24052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:19:14.601500 24052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.131.keytab
--principal=kudu/127.25.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:44573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:14.603000 24052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:14.604379 24052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:14.618213 24059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:14.618257 24060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:14.618213 24064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:14.619801 24052 server_base.cc:1034] running on GCE node
I20260806 01:19:14.620519 24052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:14.621634 24052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:14.622891 24052 hybrid_clock.cc:648] HybridClock initialized: now 1785979154622833 us; error 49 us; skew 500 ppm
I20260806 01:19:14.623260 24052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:14 dist-test-slave-7nm7 krb5kdc[23511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979154, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:14.627055 24052 init.cc:377] Logged in from keytab as kudu/127.25.236.131@KRBTEST.COM (short username kudu)
I20260806 01:19:14.629477 24052 webserver.cc:460] Webserver started at http://127.25.236.131:41213/ using document root <none> and password file <none>
I20260806 01:19:14.630667 24052 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:14.630862 24052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:14.631235 24052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:14.634719 24052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "d73db4cf1cf74820a7a99047a10cf783"
format_stamp: "Formatted at 2026-08-06 01:19:14 on dist-test-slave-7nm7"
server_key: "959cc86076eb0c4d5e58ca56e618fb04"
server_key_iv: "e1fee79d0a87ee09437a9c791172d12e"
server_key_version: "encryptionkey@0"
I20260806 01:19:14.635578 24052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "d73db4cf1cf74820a7a99047a10cf783"
format_stamp: "Formatted at 2026-08-06 01:19:14 on dist-test-slave-7nm7"
server_key: "959cc86076eb0c4d5e58ca56e618fb04"
server_key_iv: "e1fee79d0a87ee09437a9c791172d12e"
server_key_version: "encryptionkey@0"
I20260806 01:19:14.641480 24052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:19:14.645996 24074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:14.647950 24052 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:19:14.648150 24052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "d73db4cf1cf74820a7a99047a10cf783"
format_stamp: "Formatted at 2026-08-06 01:19:14 on dist-test-slave-7nm7"
server_key: "959cc86076eb0c4d5e58ca56e618fb04"
server_key_iv: "e1fee79d0a87ee09437a9c791172d12e"
server_key_version: "encryptionkey@0"
I20260806 01:19:14.648519 24052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:14.677129 24052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:14.682629 24052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:14.682993 24052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:14.684230 24052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:19:14.686146 24052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:19:14.686480 24052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 01:19:14.686709 24052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:19:14.686794 24052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:14.729663 24052 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:35555
I20260806 01:19:14.729727 24221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:35555 every 8 connection(s)
I20260806 01:19:14.732328 24052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:19:14.742676 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 24052
I20260806 01:19:14.742880 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:19:14.743510 26546 external_mini_cluster.cc:1428] Setting key bfb6e24a5cc126677472e07ccc32d12e
Aug 06 01:19:14 dist-test-slave-7nm7 krb5kdc[23511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979154, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.131@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:19:14.755193 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:14.735481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:38167 (local address 127.25.236.190:44573)
0806 01:19:14.735942 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:14.735949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:14.736940 (+ 991us) server_negotiation.cc:407] Connection header received
0806 01:19:14.738845 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:14.738849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:14.738917 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:14.739017 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:14.740478 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.741266 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.742916 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.743526 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.743686 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:14.748273 (+ 4587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:14.748278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:14.748281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:14.748322 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:14.751808 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:14.752709 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:14.752713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:14.752715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:14.752837 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:14.753348 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:14.753352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:14.753354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:14.753641 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:14.753797 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:14.754571 (+ 774us) server_negotiation.cc:299] Negotiation successful
0806 01:19:14.754793 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":229,"threads_started":1}
I20260806 01:19:14.756232 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:14.736058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:44573 (local address 127.25.236.131:38167)
0806 01:19:14.736775 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:14.736821 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:14.738593 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:14.739222 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:14.739229 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:14.739726 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:19:14.740300 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:14.740317 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.741409 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.741413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:14.742717 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:14.742735 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.743798 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.743803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:14.743999 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:14.744896 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:19:14.744927 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:19:14.748005 (+ 3078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:19:14.751971 (+ 3966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:19:14.751980 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:19:14.751997 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:19:14.752575 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:19:14.752963 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:19:14.752968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:19:14.752972 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:19:14.753178 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:19:14.753772 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:19:14.753780 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:19:14.754130 (+ 350us) client_negotiation.cc:769] Sending connection context
0806 01:19:14.754466 (+ 336us) client_negotiation.cc:240] Negotiation successful
0806 01:19:14.754984 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":301,"threads_started":1}
I20260806 01:19:14.757849 24222 heartbeater.cc:347] Connected to a master server at 127.25.236.190:44573
I20260806 01:19:14.758175 24222 heartbeater.cc:464] Registering TS with master...
I20260806 01:19:14.759157 24222 heartbeater.cc:511] Master 127.25.236.190:44573 requested a full tablet report, sending...
I20260806 01:19:14.760761 23584 ts_manager.cc:194] Registered new tserver with Master: d73db4cf1cf74820a7a99047a10cf783 (127.25.236.131:35555)
I20260806 01:19:14.761617 23584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.236.131@KRBTEST.COM'} at 127.25.236.131:38167
I20260806 01:19:14.772728 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:19:14.790833 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:14.777537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.25.236.190:44573)
0806 01:19:14.777763 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:14.777772 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:14.778189 (+ 417us) server_negotiation.cc:407] Connection header received
0806 01:19:14.778385 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:14.778389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:14.778464 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:14.778545 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:14.780052 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.780873 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.782542 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.783147 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.783262 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:14.785753 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:14.785758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:14.785762 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:14.785793 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:14.788388 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:14.789024 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:14.789028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:14.789030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:14.789150 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:14.789496 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:14.789499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:14.789501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:14.789811 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:14.790001 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:14.790345 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 01:19:14.790513 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:19:14.794777 23584 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:19:14.801968 23584 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:19:14.838615 24245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:14.815547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:35555 (local address 127.0.0.1:57754)
0806 01:19:14.828078 (+ 12531us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:14.828131 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:14.828407 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:14.829613 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:14.829621 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:14.829677 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:14.830262 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:14.830279 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.832256 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.832264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:14.834795 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:14.834817 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.837128 (+ 2311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.837135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:14.837362 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:14.837412 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 01:19:14.837563 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 01:19:14.837797 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12274,"mutex_wait_us":107,"spinlock_wait_cycles":896,"thread_start_us":12093,"threads_started":1}
I20260806 01:19:14.839618 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:14.815860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57754 (local address 127.25.236.131:35555)
0806 01:19:14.826250 (+ 10390us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:14.826264 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:14.828240 (+ 1976us) server_negotiation.cc:407] Connection header received
0806 01:19:14.828695 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:14.828703 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:14.828971 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:14.829167 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:14.830505 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.832003 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.835028 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.836680 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.836868 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:14.837042 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:14.839057 (+ 2015us) server_negotiation.cc:299] Negotiation successful
0806 01:19:14.839257 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10089,"thread_start_us":10390,"threads_started":1}
I20260806 01:19:14.839650 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:14.815713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:33105 (local address 127.0.0.1:32794)
0806 01:19:14.828670 (+ 12957us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:14.828689 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:14.828850 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:14.829417 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:14.829427 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:14.829457 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:14.829922 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:14.829937 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.831980 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.831989 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:14.835575 (+ 3586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:14.835593 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.838728 (+ 3135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.838735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:14.838910 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:14.838941 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 01:19:14.839040 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 01:19:14.839171 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12823,"mutex_wait_us":435,"spinlock_wait_cycles":8448,"thread_start_us":11687,"threads_started":1}
I20260806 01:19:14.840345 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:14.816003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32794 (local address 127.25.236.129:33105)
0806 01:19:14.827124 (+ 11121us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:14.827136 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:14.828804 (+ 1668us) server_negotiation.cc:407] Connection header received
0806 01:19:14.829017 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:14.829025 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:14.829206 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:14.829406 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:14.830119 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.831825 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.835784 (+ 3959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.837833 (+ 2049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.838036 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:14.838538 (+ 502us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:14.839805 (+ 1267us) server_negotiation.cc:299] Negotiation successful
0806 01:19:14.839997 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10910,"thread_start_us":10712,"threads_started":1}
I20260806 01:19:14.843976 23754 tablet_service.cc:1467] Processing CreateTablet for tablet 44ec0055d4d949e9a20ccdd87e9dc3a2 (DEFAULT_TABLE table=test-table [id=d87d837c5f424906beaaccee73cb1550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:14.844676 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:14.828712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50662 (local address 127.25.236.130:38267)
0806 01:19:14.835806 (+ 7094us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:14.835814 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:14.836557 (+ 743us) server_negotiation.cc:407] Connection header received
0806 01:19:14.836877 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:14.836885 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:14.837137 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:14.837368 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:14.837947 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.839404 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.841851 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.843502 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.843690 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:14.843858 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:14.844171 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 01:19:14.844334 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6940,"thread_start_us":6762,"threads_started":1}
I20260806 01:19:14.843842 24146 tablet_service.cc:1467] Processing CreateTablet for tablet 44ec0055d4d949e9a20ccdd87e9dc3a2 (DEFAULT_TABLE table=test-table [id=d87d837c5f424906beaaccee73cb1550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:14.845005 24251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:14.833055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:38267 (local address 127.0.0.1:50662)
0806 01:19:14.836286 (+ 3231us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:14.836352 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:14.836656 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:14.837354 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:14.837360 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:14.837390 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:14.837792 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:14.837802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.839587 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.839593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:14.841666 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:14.841683 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:14.843745 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:14.843751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:14.843939 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:14.843983 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 01:19:14.844095 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 01:19:14.844600 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2918,"thread_start_us":2773,"threads_started":1}
I20260806 01:19:14.845714 23754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ec0055d4d949e9a20ccdd87e9dc3a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:14.845839 24146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ec0055d4d949e9a20ccdd87e9dc3a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:14.848783 23954 tablet_service.cc:1467] Processing CreateTablet for tablet 44ec0055d4d949e9a20ccdd87e9dc3a2 (DEFAULT_TABLE table=test-table [id=d87d837c5f424906beaaccee73cb1550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:14.850091 23954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ec0055d4d949e9a20ccdd87e9dc3a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:14.858563 24253 tablet_bootstrap.cc:492] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783: Bootstrap starting.
I20260806 01:19:14.861660 24254 tablet_bootstrap.cc:492] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd: Bootstrap starting.
I20260806 01:19:14.862038 24253 tablet_bootstrap.cc:654] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:14.863817 24253 log.cc:824] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:14.865162 24255 tablet_bootstrap.cc:492] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7: Bootstrap starting.
I20260806 01:19:14.867076 24254 tablet_bootstrap.cc:654] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:14.868889 24254 log.cc:824] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:14.869451 24255 tablet_bootstrap.cc:654] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:14.870661 24255 log.cc:824] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:14.872591 24254 tablet_bootstrap.cc:492] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd: No bootstrap required, opened a new log
I20260806 01:19:14.872802 24253 tablet_bootstrap.cc:492] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783: No bootstrap required, opened a new log
I20260806 01:19:14.872889 24254 ts_tablet_manager.cc:1397] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 01:19:14.873353 24253 ts_tablet_manager.cc:1397] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20260806 01:19:14.873694 24255 tablet_bootstrap.cc:492] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7: No bootstrap required, opened a new log
I20260806 01:19:14.873955 24255 ts_tablet_manager.cc:1397] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260806 01:19:14.881690 24253 raft_consensus.cc:348] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d73db4cf1cf74820a7a99047a10cf783" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35555 } } peers { permanent_uuid: "c335dbd912594634822e8db891b48fdd" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33105 } } peers { permanent_uuid: "a7fef7fa825b4600a4d6a34418fbf6c7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38267 } }
I20260806 01:19:14.882117 24253 raft_consensus.cc:374] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:14.882323 24253 raft_consensus.cc:729] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d73db4cf1cf74820a7a99047a10cf783, State: Initialized, Role: FOLLOWER
I20260806 01:19:14.883440 24253 consensus_queue.cc:260] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d73db4cf1cf74820a7a99047a10cf783" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35555 } } peers { permanent_uuid: "c335dbd912594634822e8db891b48fdd" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33105 } } peers { permanent_uuid: "a7fef7fa825b4600a4d6a34418fbf6c7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38267 } }
I20260806 01:19:14.884294 24254 raft_consensus.cc:348] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d73db4cf1cf74820a7a99047a10cf783" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35555 } } peers { permanent_uuid: "c335dbd912594634822e8db891b48fdd" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33105 } } peers { permanent_uuid: "a7fef7fa825b4600a4d6a34418fbf6c7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38267 } }
I20260806 01:19:14.884807 24254 raft_consensus.cc:374] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:14.884966 24254 raft_consensus.cc:729] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c335dbd912594634822e8db891b48fdd, State: Initialized, Role: FOLLOWER
I20260806 01:19:14.885111 24222 heartbeater.cc:503] Master 127.25.236.190:44573 was elected leader, sending a full tablet report...
I20260806 01:19:14.884975 24255 raft_consensus.cc:348] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d73db4cf1cf74820a7a99047a10cf783" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35555 } } peers { permanent_uuid: "c335dbd912594634822e8db891b48fdd" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33105 } } peers { permanent_uuid: "a7fef7fa825b4600a4d6a34418fbf6c7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38267 } }
I20260806 01:19:14.885481 24255 raft_consensus.cc:374] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:14.885600 24255 raft_consensus.cc:729] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7fef7fa825b4600a4d6a34418fbf6c7, State: Initialized, Role: FOLLOWER
I20260806 01:19:14.885668 24254 consensus_queue.cc:260] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d73db4cf1cf74820a7a99047a10cf783" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35555 } } peers { permanent_uuid: "c335dbd912594634822e8db891b48fdd" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33105 } } peers { permanent_uuid: "a7fef7fa825b4600a4d6a34418fbf6c7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38267 } }
I20260806 01:19:14.886090 24253 ts_tablet_manager.cc:1428] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260806 01:19:14.886302 24255 consensus_queue.cc:260] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d73db4cf1cf74820a7a99047a10cf783" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35555 } } peers { permanent_uuid: "c335dbd912594634822e8db891b48fdd" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33105 } } peers { permanent_uuid: "a7fef7fa825b4600a4d6a34418fbf6c7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38267 } }
I20260806 01:19:14.888028 24045 heartbeater.cc:503] Master 127.25.236.190:44573 was elected leader, sending a full tablet report...
I20260806 01:19:14.888444 24255 ts_tablet_manager.cc:1428] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260806 01:19:14.890794 24254 ts_tablet_manager.cc:1428] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd: Time spent starting tablet: real 0.018s user 0.015s sys 0.005s
W20260806 01:19:14.984422 24224 tablet.cc:2369] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:19:14.991508 23845 tablet.cc:2369] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:19:15.092928 24264 raft_consensus.cc:482] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:19:15.093366 24264 raft_consensus.cc:504] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d73db4cf1cf74820a7a99047a10cf783" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35555 } } peers { permanent_uuid: "c335dbd912594634822e8db891b48fdd" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33105 } } peers { permanent_uuid: "a7fef7fa825b4600a4d6a34418fbf6c7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38267 } }
I20260806 01:19:15.095216 24264 leader_election.cc:288] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c335dbd912594634822e8db891b48fdd (127.25.236.129:33105), a7fef7fa825b4600a4d6a34418fbf6c7 (127.25.236.130:38267)
W20260806 01:19:15.102072 24047 tablet.cc:2369] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:19:15.102454 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:15.095910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:33105 (local address 127.25.236.131:34325)
0806 01:19:15.096155 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:15.096195 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:15.096353 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:15.096977 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:15.096984 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:15.097014 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:15.097250 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:15.097265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.098580 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.098589 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:15.100398 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:15.100410 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.101517 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.101523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:15.101776 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:15.101817 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 01:19:15.101921 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 01:19:15.102102 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":28}
I20260806 01:19:15.103057 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:15.096189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:34325 (local address 127.25.236.129:33105)
0806 01:19:15.096422 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:15.096429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:15.096452 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:19:15.096534 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:15.096543 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:15.096638 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:15.096763 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:15.097412 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.098447 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.100556 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.101358 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.101533 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:15.101620 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:15.102661 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0806 01:19:15.102805 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 01:19:15.105020 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:15.096823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:40747 (local address 127.25.236.130:38267)
0806 01:19:15.097044 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:15.097051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:15.097527 (+ 476us) server_negotiation.cc:407] Connection header received
0806 01:19:15.097774 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:15.097780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:15.097856 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:15.097952 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:15.098709 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.099845 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.102535 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.103891 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.104052 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:15.104120 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:15.104608 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 01:19:15.104768 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 01:19:15.104818 23787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ec0055d4d949e9a20ccdd87e9dc3a2" candidate_uuid: "d73db4cf1cf74820a7a99047a10cf783" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c335dbd912594634822e8db891b48fdd" is_pre_election: true
I20260806 01:19:15.105129 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:15.096462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:38267 (local address 127.25.236.131:40747)
0806 01:19:15.097408 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:15.097425 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:15.097631 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:15.097990 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:15.097994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:15.098018 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:15.098467 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:15.098483 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.100013 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.100022 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:15.102334 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:15.102351 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.104226 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.104238 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:15.104422 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:15.104448 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 01:19:15.104570 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 01:19:15.104695 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":4,"thread_start_us":593,"threads_started":1}
I20260806 01:19:15.105399 23787 raft_consensus.cc:2454] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d73db4cf1cf74820a7a99047a10cf783 in term 0.
I20260806 01:19:15.106171 24075 leader_election.cc:302] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c335dbd912594634822e8db891b48fdd, d73db4cf1cf74820a7a99047a10cf783; no voters:
I20260806 01:19:15.106455 23984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ec0055d4d949e9a20ccdd87e9dc3a2" candidate_uuid: "d73db4cf1cf74820a7a99047a10cf783" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7fef7fa825b4600a4d6a34418fbf6c7" is_pre_election: true
I20260806 01:19:15.106680 24264 raft_consensus.cc:2781] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:19:15.106827 24264 raft_consensus.cc:482] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:19:15.106961 24264 raft_consensus.cc:3037] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:15.106997 23984 raft_consensus.cc:2454] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d73db4cf1cf74820a7a99047a10cf783 in term 0.
I20260806 01:19:15.109652 24264 raft_consensus.cc:504] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d73db4cf1cf74820a7a99047a10cf783" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35555 } } peers { permanent_uuid: "c335dbd912594634822e8db891b48fdd" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33105 } } peers { permanent_uuid: "a7fef7fa825b4600a4d6a34418fbf6c7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38267 } }
I20260806 01:19:15.110370 24264 leader_election.cc:288] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [CANDIDATE]: Term 1 election: Requested vote from peers c335dbd912594634822e8db891b48fdd (127.25.236.129:33105), a7fef7fa825b4600a4d6a34418fbf6c7 (127.25.236.130:38267)
I20260806 01:19:15.110734 23787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ec0055d4d949e9a20ccdd87e9dc3a2" candidate_uuid: "d73db4cf1cf74820a7a99047a10cf783" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c335dbd912594634822e8db891b48fdd"
I20260806 01:19:15.110867 23984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ec0055d4d949e9a20ccdd87e9dc3a2" candidate_uuid: "d73db4cf1cf74820a7a99047a10cf783" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7fef7fa825b4600a4d6a34418fbf6c7"
I20260806 01:19:15.110957 23787 raft_consensus.cc:3037] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:15.111078 23984 raft_consensus.cc:3037] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:15.113758 23787 raft_consensus.cc:2454] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d73db4cf1cf74820a7a99047a10cf783 in term 1.
I20260806 01:19:15.114161 24075 leader_election.cc:302] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c335dbd912594634822e8db891b48fdd, d73db4cf1cf74820a7a99047a10cf783; no voters:
I20260806 01:19:15.114398 23984 raft_consensus.cc:2454] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d73db4cf1cf74820a7a99047a10cf783 in term 1.
I20260806 01:19:15.114535 24264 raft_consensus.cc:2781] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:15.115082 24264 raft_consensus.cc:686] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [term 1 LEADER]: Becoming Leader. State: Replica: d73db4cf1cf74820a7a99047a10cf783, State: Running, Role: LEADER
I20260806 01:19:15.115571 24264 consensus_queue.cc:237] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d73db4cf1cf74820a7a99047a10cf783" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35555 } } peers { permanent_uuid: "c335dbd912594634822e8db891b48fdd" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33105 } } peers { permanent_uuid: "a7fef7fa825b4600a4d6a34418fbf6c7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38267 } }
I20260806 01:19:15.121295 23584 catalog_manager.cc:5495] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 reported cstate change: term changed from 0 to 1, leader changed from <none> to d73db4cf1cf74820a7a99047a10cf783 (127.25.236.131). New cstate: current_term: 1 leader_uuid: "d73db4cf1cf74820a7a99047a10cf783" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d73db4cf1cf74820a7a99047a10cf783" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 35555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c335dbd912594634822e8db891b48fdd" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7fef7fa825b4600a4d6a34418fbf6c7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38267 } health_report { overall_health: UNKNOWN } } }
I20260806 01:19:15.196854 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:15.191052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57762 (local address 127.25.236.131:35555)
0806 01:19:15.191244 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:15.191249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:15.191291 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:19:15.191486 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:15.191489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:15.191566 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:15.191704 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:19:15.192031 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.192866 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.194437 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.195142 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.195243 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:15.195756 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:19:15.196337 (+ 581us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:19:15.196460 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:15.196573 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 01:19:15.196654 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:19:15.205158 23984 raft_consensus.cc:1261] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7 [term 1 FOLLOWER]: Refusing update from remote peer d73db4cf1cf74820a7a99047a10cf783: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:19:15.205178 23787 raft_consensus.cc:1261] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd [term 1 FOLLOWER]: Refusing update from remote peer d73db4cf1cf74820a7a99047a10cf783: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:19:15.206290 24272 consensus_queue.cc:1035] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7fef7fa825b4600a4d6a34418fbf6c7" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:19:15.206687 24264 consensus_queue.cc:1035] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c335dbd912594634822e8db891b48fdd" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:19:15.218006 24275 mvcc.cc:205] Tried to move back new op lower bound from 7315370619710230528 to 7315370619356925952. Current Snapshot: MvccSnapshot[applied={T|T < 7315370619710230528}]
I20260806 01:19:15.220469 24276 mvcc.cc:205] Tried to move back new op lower bound from 7315370619710230528 to 7315370619356925952. Current Snapshot: MvccSnapshot[applied={T|T < 7315370619710230528}]
I20260806 01:19:15.225338 24278 mvcc.cc:205] Tried to move back new op lower bound from 7315370619710230528 to 7315370619356925952. Current Snapshot: MvccSnapshot[applied={T|T < 7315370619710230528}]
I20260806 01:19:15.233175 23581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:19:15.233498 23581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:19:15.237915 23581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a3aa09e108540c3b6d870b04bfbb7aa: Sending DeleteTablet for 3 replicas of tablet 44ec0055d4d949e9a20ccdd87e9dc3a2
I20260806 01:19:15.239153 24146 tablet_service.cc:1514] Processing DeleteTablet for tablet 44ec0055d4d949e9a20ccdd87e9dc3a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:15 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:57754
I20260806 01:19:15.239238 23754 tablet_service.cc:1514] Processing DeleteTablet for tablet 44ec0055d4d949e9a20ccdd87e9dc3a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:15 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:32794
I20260806 01:19:15.240566 23954 tablet_service.cc:1514] Processing DeleteTablet for tablet 44ec0055d4d949e9a20ccdd87e9dc3a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:15 UTC) from {username='kudu', principal='kudu/127.25.236.190@KRBTEST.COM'} at 127.0.0.1:50662
I20260806 01:19:15.249581 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 23663
I20260806 01:19:15.259778 24291 tablet_replica.cc:330] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7: stopping tablet replica
I20260806 01:19:15.259989 24289 tablet_replica.cc:330] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783: stopping tablet replica
I20260806 01:19:15.260479 24291 raft_consensus.cc:2229] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:19:15.260488 24289 raft_consensus.cc:2229] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:19:15.261288 24291 raft_consensus.cc:2258] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:15.261489 24290 tablet_replica.cc:330] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd: stopping tablet replica
I20260806 01:19:15.261567 24289 raft_consensus.cc:2258] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:15.262073 24290 raft_consensus.cc:2229] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:19:15.262734 24290 raft_consensus.cc:2258] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:15.264058 24289 ts_tablet_manager.cc:1905] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:15.264801 24290 ts_tablet_manager.cc:1905] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:15.264853 24291 ts_tablet_manager.cc:1905] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 01:19:15 dist-test-slave-7nm7 krb5kdc[23511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.129@KRBTEST.COM
I20260806 01:19:15.272300 24290 ts_tablet_manager.cc:1918] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:15.272544 24290 log.cc:1196] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/44ec0055d4d949e9a20ccdd87e9dc3a2
I20260806 01:19:15.272744 24289 ts_tablet_manager.cc:1918] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:15.273324 24290 ts_tablet_manager.cc:1939] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P c335dbd912594634822e8db891b48fdd: Deleting consensus metadata
I20260806 01:19:15.274333 24289 log.cc:1196] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/44ec0055d4d949e9a20ccdd87e9dc3a2
I20260806 01:19:15.275039 24289 ts_tablet_manager.cc:1939] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P d73db4cf1cf74820a7a99047a10cf783: Deleting consensus metadata
I20260806 01:19:15.275104 24291 ts_tablet_manager.cc:1918] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:15.275275 24291 log.cc:1196] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/44ec0055d4d949e9a20ccdd87e9dc3a2
I20260806 01:19:15.275853 24291 ts_tablet_manager.cc:1939] T 44ec0055d4d949e9a20ccdd87e9dc3a2 P a7fef7fa825b4600a4d6a34418fbf6c7: Deleting consensus metadata
I20260806 01:19:15.275866 23558 catalog_manager.cc:4841] TS c335dbd912594634822e8db891b48fdd (127.25.236.129:33105): tablet 44ec0055d4d949e9a20ccdd87e9dc3a2 (table test-table [id=d87d837c5f424906beaaccee73cb1550]) successfully deleted
I20260806 01:19:15.278301 23558 catalog_manager.cc:4841] TS a7fef7fa825b4600a4d6a34418fbf6c7 (127.25.236.130:38267): tablet 44ec0055d4d949e9a20ccdd87e9dc3a2 (table test-table [id=d87d837c5f424906beaaccee73cb1550]) successfully deleted
I20260806 01:19:15.278684 23563 catalog_manager.cc:4841] TS d73db4cf1cf74820a7a99047a10cf783 (127.25.236.131:35555): tablet 44ec0055d4d949e9a20ccdd87e9dc3a2 (table test-table [id=d87d837c5f424906beaaccee73cb1550]) successfully deleted
I20260806 01:19:15.283272 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:15.252388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.25.236.129:33105)
0806 01:19:15.252581 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:15.252588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:15.252607 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:19:15.252669 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:15.252674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:15.252764 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:15.252868 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:15.256959 (+ 4091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.258154 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.261274 (+ 3120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.262029 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.262173 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:15.275760 (+ 13587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:15.275777 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:15.275795 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:15.275846 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:15.280097 (+ 4251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:15.280907 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:15.280914 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:15.280916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:15.281047 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:15.281472 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:15.281477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:15.281481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:15.281849 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:15.282177 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:15.282530 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 01:19:15.282800 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:19:15.284332 23837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:15.384625 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 23663
I20260806 01:19:15.420413 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 23855
Aug 06 01:19:15 dist-test-slave-7nm7 krb5kdc[23511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.130@KRBTEST.COM
I20260806 01:19:15.439620 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:15.421306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50676 (local address 127.25.236.130:38267)
0806 01:19:15.421487 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:15.421492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:15.421510 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:19:15.421651 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:15.421655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:15.421728 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:15.421811 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:15.423668 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.424462 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.426597 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.426999 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.427116 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:15.432583 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:15.432603 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:15.432625 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:15.432672 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:15.436498 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:15.437251 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:15.437261 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:15.437265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:15.437425 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:15.437861 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:15.437867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:15.437871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:15.438375 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:15.438694 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:15.438959 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 01:19:15.439218 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:19:15.440368 24040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:15.533519 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 23855
I20260806 01:19:15.559139 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 24052
Aug 06 01:19:15 dist-test-slave-7nm7 krb5kdc[23511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.131@KRBTEST.COM
I20260806 01:19:15.583572 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:15.560095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57764 (local address 127.25.236.131:35555)
0806 01:19:15.560265 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:15.560269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:15.560287 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:19:15.560434 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:15.560438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:15.560503 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:15.560585 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:15.562471 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.563322 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.565309 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:15.565792 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:15.565906 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:15.572243 (+ 6337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:15.572279 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:15.572300 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:15.572360 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:15.579290 (+ 6930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:15.580501 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:15.580515 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:15.580521 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:15.580767 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:15.581253 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:15.581259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:15.581262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:15.581766 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:15.582281 (+ 515us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:15.582775 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 01:19:15.583034 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 01:19:15.584726 24217 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:15.715091 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 24052
I20260806 01:19:15.746860 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 23530
I20260806 01:19:15.748072 23619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:15.865455 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 23530
2026-08-06T01:19:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (6389 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T01:19:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:15Z Disabled control of system clock
I20260806 01:19:15.973273 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:45947
--webserver_interface=127.25.236.190
--webserver_port=0
--builtin_ntp_servers=127.25.236.148:41475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:45947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:19:16.361248 24304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:16.361640 24304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:16.361753 24304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:16.369943 24304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:16.370077 24304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:16.370118 24304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:16.370172 24304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:16.370217 24304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:16.380251 24304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:45947
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:45947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:16.381634 24304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:16.383486 24304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:16.396071 24311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:16.396070 24314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:16.396082 24312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:16.396905 24304 server_base.cc:1034] running on GCE node
I20260806 01:19:16.397738 24304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:16.399394 24304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:16.400657 24304 hybrid_clock.cc:648] HybridClock initialized: now 1785979156400584 us; error 60 us; skew 500 ppm
I20260806 01:19:16.401036 24304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:19:16.403805 24304 webserver.cc:460] Webserver started at http://127.25.236.190:33893/ using document root <none> and password file <none>
I20260806 01:19:16.404537 24304 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:16.404675 24304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:16.405045 24304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:16.408087 24304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "9266564eba9c44ab8334a6baf94718dd"
format_stamp: "Formatted at 2026-08-06 01:19:16 on dist-test-slave-7nm7"
server_key: "8f2d2b052f9a3fe366431ccd872c744c"
server_key_iv: "e0d86f48b50825c6e5ea21dae91459ac"
server_key_version: "encryptionkey@0"
I20260806 01:19:16.408918 24304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "9266564eba9c44ab8334a6baf94718dd"
format_stamp: "Formatted at 2026-08-06 01:19:16 on dist-test-slave-7nm7"
server_key: "8f2d2b052f9a3fe366431ccd872c744c"
server_key_iv: "e0d86f48b50825c6e5ea21dae91459ac"
server_key_version: "encryptionkey@0"
I20260806 01:19:16.414041 24304 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:19:16.417748 24324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:16.419036 24304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:16.419231 24304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "9266564eba9c44ab8334a6baf94718dd"
format_stamp: "Formatted at 2026-08-06 01:19:16 on dist-test-slave-7nm7"
server_key: "8f2d2b052f9a3fe366431ccd872c744c"
server_key_iv: "e0d86f48b50825c6e5ea21dae91459ac"
server_key_version: "encryptionkey@0"
I20260806 01:19:16.419451 24304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:16.444799 24304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:16.446038 24304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:16.446530 24304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:16.466286 24304 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:45947
I20260806 01:19:16.466342 24399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:45947 every 8 connection(s)
I20260806 01:19:16.467621 24304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:16.471679 24400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:16.473174 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 24304
I20260806 01:19:16.473363 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:16.473944 26546 external_mini_cluster.cc:1428] Setting key a507012f05b015c94c6936e7ad065e66
I20260806 01:19:16.485095 24400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd: Bootstrap starting.
I20260806 01:19:16.487269 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:16.476000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52560 (local address 127.25.236.190:45947)
0806 01:19:16.477044 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:16.477055 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:16.477098 (+ 43us) server_negotiation.cc:407] Connection header received
0806 01:19:16.478873 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:16.478893 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:16.479335 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:16.479717 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:16.480268 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:16.481484 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:16.482923 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:16.483590 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:16.483778 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:16.485012 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:16.485020 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:16.485032 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:16.485604 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:16.485675 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:16.485696 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:16.485821 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:16.486036 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 01:19:16.486391 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":372,"threads_started":1}
I20260806 01:19:16.488642 24400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:16.489996 24400 log.cc:824] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:16.492679 24400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd: No bootstrap required, opened a new log
I20260806 01:19:16.500370 24400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9266564eba9c44ab8334a6baf94718dd" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 45947 } }
I20260806 01:19:16.500727 24400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:16.500800 24400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9266564eba9c44ab8334a6baf94718dd, State: Initialized, Role: FOLLOWER
I20260806 01:19:16.501304 24400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9266564eba9c44ab8334a6baf94718dd" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 45947 } }
I20260806 01:19:16.501595 24400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:16.501803 24400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:16.502008 24400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:16.504241 24400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9266564eba9c44ab8334a6baf94718dd" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 45947 } }
I20260806 01:19:16.504773 24400 leader_election.cc:302] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9266564eba9c44ab8334a6baf94718dd; no voters:
I20260806 01:19:16.505395 24400 leader_election.cc:288] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:16.505573 24406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:16.506503 24406 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [term 1 LEADER]: Becoming Leader. State: Replica: 9266564eba9c44ab8334a6baf94718dd, State: Running, Role: LEADER
I20260806 01:19:16.507098 24406 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9266564eba9c44ab8334a6baf94718dd" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 45947 } }
I20260806 01:19:16.507688 24400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:16.510269 24409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9266564eba9c44ab8334a6baf94718dd. Latest consensus state: current_term: 1 leader_uuid: "9266564eba9c44ab8334a6baf94718dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9266564eba9c44ab8334a6baf94718dd" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 45947 } } }
I20260806 01:19:16.510290 24407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9266564eba9c44ab8334a6baf94718dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9266564eba9c44ab8334a6baf94718dd" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 45947 } } }
I20260806 01:19:16.510756 24409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:16.510794 24407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:16.511740 24414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:16.516875 24414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:16.529039 24414 catalog_manager.cc:1347] Generated new cluster ID: b393d82261a240809ad6cba867c03746
I20260806 01:19:16.529177 24414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:16.548837 24414 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:16.550706 24414 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:16.567308 24414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd: Generated new TSK 0
I20260806 01:19:16.568463 24414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:16.573251 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.129:0
--local_ip_for_outbound_sockets=127.25.236.129
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:45947
--builtin_ntp_servers=127.25.236.148:41475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:19:16.955042 24432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:16.955309 24432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:16.955395 24432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:16.962399 24432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:16.962495 24432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:16.962604 24432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.129
I20260806 01:19:16.971300 24432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.236.129
--webserver_port=0
--tserver_master_addrs=127.25.236.190:45947
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.129
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:16.972463 24432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:16.974115 24432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:16.987319 24440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:16.987470 24439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:16.987640 24444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:16.988623 24432 server_base.cc:1034] running on GCE node
I20260806 01:19:16.989257 24432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:16.990284 24432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:16.991510 24432 hybrid_clock.cc:648] HybridClock initialized: now 1785979156991470 us; error 37 us; skew 500 ppm
I20260806 01:19:16.991842 24432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:19:16.994223 24432 webserver.cc:460] Webserver started at http://127.25.236.129:34557/ using document root <none> and password file <none>
I20260806 01:19:16.994988 24432 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:16.995081 24432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:16.995541 24432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:16.998672 24432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/data/instance:
uuid: "419ed089854243b48be4ae1b6b0a3891"
format_stamp: "Formatted at 2026-08-06 01:19:16 on dist-test-slave-7nm7"
server_key: "a7dc5f1829a5bd691db572d183f1a494"
server_key_iv: "24dcfd449eed9975eda78d29fb9d5c5a"
server_key_version: "encryptionkey@0"
I20260806 01:19:16.999488 24432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance:
uuid: "419ed089854243b48be4ae1b6b0a3891"
format_stamp: "Formatted at 2026-08-06 01:19:16 on dist-test-slave-7nm7"
server_key: "a7dc5f1829a5bd691db572d183f1a494"
server_key_iv: "24dcfd449eed9975eda78d29fb9d5c5a"
server_key_version: "encryptionkey@0"
I20260806 01:19:17.005218 24432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:19:17.008955 24457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:17.010479 24432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:19:17.010650 24432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/wal
uuid: "419ed089854243b48be4ae1b6b0a3891"
format_stamp: "Formatted at 2026-08-06 01:19:16 on dist-test-slave-7nm7"
server_key: "a7dc5f1829a5bd691db572d183f1a494"
server_key_iv: "24dcfd449eed9975eda78d29fb9d5c5a"
server_key_version: "encryptionkey@0"
I20260806 01:19:17.010859 24432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:17.050885 24432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:17.051761 24432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:17.052085 24432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:17.053112 24432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:19:17.054909 24432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:19:17.055006 24432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:17.055433 24432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:19:17.055523 24432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:17.094974 24432 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.129:35797
I20260806 01:19:17.095034 24619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.129:35797 every 8 connection(s)
I20260806 01:19:17.096359 24432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/data/info.pb
I20260806 01:19:17.098960 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 24432
I20260806 01:19:17.099351 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/wal/instance
I20260806 01:19:17.100136 26546 external_mini_cluster.cc:1428] Setting key 8df67532038f9743379f58fba9db8ebe
I20260806 01:19:17.103749 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.130:0
--local_ip_for_outbound_sockets=127.25.236.130
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:45947
--builtin_ntp_servers=127.25.236.148:41475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:19:17.113682 24623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:17.100117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.129:42541 (local address 127.25.236.190:45947)
0806 01:19:17.101051 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:17.101061 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:17.102000 (+ 939us) server_negotiation.cc:407] Connection header received
0806 01:19:17.104715 (+ 2715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:17.104720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:17.104818 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:17.105005 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:17.106799 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:17.107728 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:17.110364 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:17.110835 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:17.110944 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:17.112763 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:17.112769 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:17.112773 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:17.112794 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:17.112845 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:17.112858 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:17.112952 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:17.113294 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 01:19:17.113385 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":785,"thread_start_us":667,"threads_started":1}
I20260806 01:19:17.115721 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:17.100803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:45947 (local address 127.25.236.129:42541)
0806 01:19:17.101818 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:17.101856 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:17.104335 (+ 2479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:17.105489 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:17.105505 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:17.105650 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:19:17.106570 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:17.106601 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:17.107950 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:17.107958 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:17.110122 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:17.110141 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:17.111098 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:17.111109 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:17.111444 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:17.112315 (+ 871us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:19:17.112320 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:19:17.112377 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:19:17.112500 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:19:17.112982 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:19:17.112996 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:19:17.113007 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 01:19:17.113229 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 01:19:17.113682 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":373,"threads_started":1}
I20260806 01:19:17.118345 24620 heartbeater.cc:347] Connected to a master server at 127.25.236.190:45947
I20260806 01:19:17.118978 24620 heartbeater.cc:464] Registering TS with master...
I20260806 01:19:17.120249 24620 heartbeater.cc:511] Master 127.25.236.190:45947 requested a full tablet report, sending...
I20260806 01:19:17.123695 24351 ts_manager.cc:194] Registered new tserver with Master: 419ed089854243b48be4ae1b6b0a3891 (127.25.236.129:35797)
I20260806 01:19:17.126006 24351 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.236.129:42541
W20260806 01:19:17.475993 24625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:17.476286 24625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:17.476437 24625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:17.486119 24625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:17.486299 24625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:17.486479 24625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.130
I20260806 01:19:17.496472 24625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.236.130
--webserver_port=0
--tserver_master_addrs=127.25.236.190:45947
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.130
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:17.497849 24625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:17.499703 24625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:17.514874 24634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:17.514853 24639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:17.514958 24630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:17.521064 24625 server_base.cc:1034] running on GCE node
I20260806 01:19:17.521894 24625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:17.523458 24625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:17.524756 24625 hybrid_clock.cc:648] HybridClock initialized: now 1785979157524687 us; error 62 us; skew 500 ppm
I20260806 01:19:17.525161 24625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:19:17.528170 24625 webserver.cc:460] Webserver started at http://127.25.236.130:43369/ using document root <none> and password file <none>
I20260806 01:19:17.528913 24625 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:17.529018 24625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:17.529284 24625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:17.532325 24625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/data/instance:
uuid: "22cd371b03ec435aadfacf233ff4ff09"
format_stamp: "Formatted at 2026-08-06 01:19:17 on dist-test-slave-7nm7"
server_key: "e4bcc5eba4c9ac133bc7a1683bc368f1"
server_key_iv: "79569a4261fa6f8579a14f546f757062"
server_key_version: "encryptionkey@0"
I20260806 01:19:17.533106 24625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance:
uuid: "22cd371b03ec435aadfacf233ff4ff09"
format_stamp: "Formatted at 2026-08-06 01:19:17 on dist-test-slave-7nm7"
server_key: "e4bcc5eba4c9ac133bc7a1683bc368f1"
server_key_iv: "79569a4261fa6f8579a14f546f757062"
server_key_version: "encryptionkey@0"
I20260806 01:19:17.538628 24625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:19:17.542359 24647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:17.544095 24625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:19:17.544323 24625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/wal
uuid: "22cd371b03ec435aadfacf233ff4ff09"
format_stamp: "Formatted at 2026-08-06 01:19:17 on dist-test-slave-7nm7"
server_key: "e4bcc5eba4c9ac133bc7a1683bc368f1"
server_key_iv: "79569a4261fa6f8579a14f546f757062"
server_key_version: "encryptionkey@0"
I20260806 01:19:17.544576 24625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:17.558910 24625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:17.560559 24625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:17.560935 24625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:17.562005 24625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:19:17.563823 24625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:19:17.563926 24625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:17.564083 24625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:19:17.564208 24625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:17.602866 24625 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.130:35175
I20260806 01:19:17.603058 24800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.130:35175 every 8 connection(s)
I20260806 01:19:17.604247 24625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/data/info.pb
I20260806 01:19:17.607887 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 24625
I20260806 01:19:17.608201 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/wal/instance
I20260806 01:19:17.609294 26546 external_mini_cluster.cc:1428] Setting key ce96efc18ee3863911ed8b4211e942db
I20260806 01:19:17.612640 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.236.131:0
--local_ip_for_outbound_sockets=127.25.236.131
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:45947
--builtin_ntp_servers=127.25.236.148:41475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:19:17.620997 24623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:17.607269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:34007 (local address 127.25.236.190:45947)
0806 01:19:17.607533 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:17.607540 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:17.609327 (+ 1787us) server_negotiation.cc:407] Connection header received
0806 01:19:17.611698 (+ 2371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:17.611704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:17.611786 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:17.611909 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:17.613015 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:17.614271 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:17.616791 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:17.617518 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:17.617680 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:17.619915 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:17.619925 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:17.619950 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:17.619983 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:17.620038 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:17.620065 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:17.620187 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:17.620459 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 01:19:17.620578 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 01:19:17.622866 24804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:17.607851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:45947 (local address 127.25.236.130:34007)
0806 01:19:17.608962 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:17.609005 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:17.611380 (+ 2375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:17.612100 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:17.612109 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:17.612198 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:19:17.612822 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:17.612851 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:17.614482 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:17.614486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:17.616592 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:17.616605 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:17.617803 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:17.617833 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:17.618335 (+ 502us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:17.619325 (+ 990us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:19:17.619331 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:19:17.619391 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:19:17.619494 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:19:17.620204 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:19:17.620216 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:19:17.620227 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 01:19:17.620442 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 01:19:17.620788 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":396,"threads_started":1}
I20260806 01:19:17.624977 24801 heartbeater.cc:347] Connected to a master server at 127.25.236.190:45947
I20260806 01:19:17.625365 24801 heartbeater.cc:464] Registering TS with master...
I20260806 01:19:17.626148 24801 heartbeater.cc:511] Master 127.25.236.190:45947 requested a full tablet report, sending...
I20260806 01:19:17.627923 24351 ts_manager.cc:194] Registered new tserver with Master: 22cd371b03ec435aadfacf233ff4ff09 (127.25.236.130:35175)
I20260806 01:19:17.628907 24351 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.236.130:34007
W20260806 01:19:17.979703 24805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:17.980000 24805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:17.980189 24805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:17.988224 24805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:17.988348 24805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:17.988525 24805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.236.131
I20260806 01:19:17.998721 24805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.236.131
--webserver_port=0
--tserver_master_addrs=127.25.236.190:45947
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.236.131
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:18.000430 24805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:18.002550 24805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:18.016067 24815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:18.016167 24813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:18.016199 24812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:18.017357 24805 server_base.cc:1034] running on GCE node
I20260806 01:19:18.018415 24805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:18.019357 24805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:18.020663 24805 hybrid_clock.cc:648] HybridClock initialized: now 1785979158020596 us; error 37 us; skew 500 ppm
I20260806 01:19:18.021056 24805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:19:18.023572 24805 webserver.cc:460] Webserver started at http://127.25.236.131:43887/ using document root <none> and password file <none>
I20260806 01:19:18.024309 24805 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:18.024398 24805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:18.024660 24805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:18.027513 24805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/data/instance:
uuid: "61b05cb34f5a4e0ea967f51f6c423c58"
format_stamp: "Formatted at 2026-08-06 01:19:18 on dist-test-slave-7nm7"
server_key: "27e832a022e35d4a860bd2e91fa5a28e"
server_key_iv: "1b866126321cb26f5398a8613b2467fa"
server_key_version: "encryptionkey@0"
I20260806 01:19:18.028249 24805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance:
uuid: "61b05cb34f5a4e0ea967f51f6c423c58"
format_stamp: "Formatted at 2026-08-06 01:19:18 on dist-test-slave-7nm7"
server_key: "27e832a022e35d4a860bd2e91fa5a28e"
server_key_iv: "1b866126321cb26f5398a8613b2467fa"
server_key_version: "encryptionkey@0"
I20260806 01:19:18.033768 24805 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:19:18.037357 24823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:18.038729 24805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 01:19:18.038909 24805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/wal
uuid: "61b05cb34f5a4e0ea967f51f6c423c58"
format_stamp: "Formatted at 2026-08-06 01:19:18 on dist-test-slave-7nm7"
server_key: "27e832a022e35d4a860bd2e91fa5a28e"
server_key_iv: "1b866126321cb26f5398a8613b2467fa"
server_key_version: "encryptionkey@0"
I20260806 01:19:18.039091 24805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:18.065570 24805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:18.066448 24805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:18.066766 24805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:18.067811 24805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:19:18.069288 24805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:19:18.069377 24805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:18.069527 24805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:19:18.069626 24805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:18.105901 24805 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.131:44185
I20260806 01:19:18.106014 24979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.131:44185 every 8 connection(s)
I20260806 01:19:18.107280 24805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/data/info.pb
I20260806 01:19:18.112989 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 24805
I20260806 01:19:18.113368 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/wal/instance
I20260806 01:19:18.113850 26546 external_mini_cluster.cc:1428] Setting key 0dc2188a08c97760ac21f8c3358f88a4
I20260806 01:19:18.121483 24623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.110207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.131:33899 (local address 127.25.236.190:45947)
0806 01:19:18.110469 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:18.110475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:18.111865 (+ 1390us) server_negotiation.cc:407] Connection header received
0806 01:19:18.113792 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:18.113798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:18.113894 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:18.114013 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:18.115483 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.116398 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.118433 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.118848 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.118959 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.120378 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:18.120383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:18.120386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:18.120405 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:18.120438 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:18.120449 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:18.120524 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:18.121047 (+ 523us) server_negotiation.cc:299] Negotiation successful
0806 01:19:18.121141 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260806 01:19:18.122371 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.110812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:45947 (local address 127.25.236.131:33899)
0806 01:19:18.111702 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:18.111748 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:18.113493 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:18.114412 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:18.114429 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:18.114548 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:19:18.115248 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:18.115291 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.116565 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.116570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:18.118046 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:18.118070 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.119071 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.119076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:18.119303 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.120021 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:19:18.120024 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:19:18.120075 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:19:18.120163 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:19:18.120573 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:19:18.120582 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:19:18.120589 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 01:19:18.120775 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 01:19:18.121203 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":275,"threads_started":1}
I20260806 01:19:18.124187 24980 heartbeater.cc:347] Connected to a master server at 127.25.236.190:45947
I20260806 01:19:18.124524 24980 heartbeater.cc:464] Registering TS with master...
I20260806 01:19:18.125316 24980 heartbeater.cc:511] Master 127.25.236.190:45947 requested a full tablet report, sending...
I20260806 01:19:18.126791 24351 ts_manager.cc:194] Registered new tserver with Master: 61b05cb34f5a4e0ea967f51f6c423c58 (127.25.236.131:44185)
I20260806 01:19:18.127905 24351 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.236.131:33899
I20260806 01:19:18.129746 24620 heartbeater.cc:503] Master 127.25.236.190:45947 was elected leader, sending a full tablet report...
I20260806 01:19:18.129968 26546 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:19:18.141664 24623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.134595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52570 (local address 127.25.236.190:45947)
0806 01:19:18.134761 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:18.134765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:18.135340 (+ 575us) server_negotiation.cc:407] Connection header received
0806 01:19:18.135490 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:18.135494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:18.135560 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:18.135658 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:18.136120 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.136820 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.138390 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.138832 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.138944 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.140889 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:18.140895 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:18.140899 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:18.140929 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:18.140966 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:18.140993 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:18.141095 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:18.141331 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 01:19:18.141412 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:19:18.147047 24351 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:19:18.155043 24351 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:19:18.184835 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.170508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34256 (local address 127.25.236.130:35175)
0806 01:19:18.171657 (+ 1149us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:18.171669 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:18.172700 (+ 1031us) server_negotiation.cc:407] Connection header received
0806 01:19:18.173129 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:18.173136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:18.173338 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:18.173491 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:18.174691 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.176697 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.180488 (+ 3791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.182570 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.182764 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.183062 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:18.184026 (+ 964us) server_negotiation.cc:299] Negotiation successful
0806 01:19:18.184286 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":971,"thread_start_us":842,"threads_started":1}
I20260806 01:19:18.184859 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.169581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:35797 (local address 127.0.0.1:39554)
0806 01:19:18.171007 (+ 1426us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:18.171075 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:18.171286 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:18.172968 (+ 1682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:18.172976 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:18.173034 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:18.173788 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:18.173808 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.176456 (+ 2648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.176465 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:18.179945 (+ 3480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:18.179967 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.183564 (+ 3597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.183571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:18.183813 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.183855 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 01:19:18.183962 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 01:19:18.184255 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1106,"mutex_wait_us":83,"thread_start_us":885,"threads_started":1}
I20260806 01:19:18.186447 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.170123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.130:35175 (local address 127.0.0.1:34256)
0806 01:19:18.172588 (+ 2465us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:18.172605 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:18.172814 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:18.173839 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:18.173847 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:18.173888 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:18.174452 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:18.174467 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.176949 (+ 2482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.176958 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:18.180202 (+ 3244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:18.180224 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.183308 (+ 3084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.183318 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:18.183665 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.183745 (+ 80us) client_negotiation.cc:769] Sending connection context
0806 01:19:18.185625 (+ 1880us) client_negotiation.cc:240] Negotiation successful
0806 01:19:18.185846 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2328,"mutex_wait_us":3,"thread_start_us":2225,"threads_started":1}
I20260806 01:19:18.187456 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.169765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39554 (local address 127.25.236.129:35797)
0806 01:19:18.172251 (+ 2486us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:18.172262 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:18.172295 (+ 33us) server_negotiation.cc:407] Connection header received
0806 01:19:18.172395 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:18.172402 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:18.172659 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:18.172851 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:18.174092 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.176229 (+ 2137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.180289 (+ 4060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.182453 (+ 2164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.182710 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.183338 (+ 628us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:18.186657 (+ 3319us) server_negotiation.cc:299] Negotiation successful
0806 01:19:18.186919 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2208,"thread_start_us":2056,"threads_started":1}
I20260806 01:19:18.192052 24712 tablet_service.cc:1467] Processing CreateTablet for tablet 96052b3c23ef49ecb7dc91201c6e65d1 (DEFAULT_TABLE table=test-table [id=0cc2767a02ee494a9467cea277d79896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:18.194074 24712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96052b3c23ef49ecb7dc91201c6e65d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:18.193511 24526 tablet_service.cc:1467] Processing CreateTablet for tablet 96052b3c23ef49ecb7dc91201c6e65d1 (DEFAULT_TABLE table=test-table [id=0cc2767a02ee494a9467cea277d79896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:18.195284 24526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96052b3c23ef49ecb7dc91201c6e65d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:18.212841 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.173504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46564 (local address 127.25.236.131:44185)
0806 01:19:18.203066 (+ 29562us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:18.203080 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:18.203269 (+ 189us) server_negotiation.cc:407] Connection header received
0806 01:19:18.203411 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:18.203419 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:18.203713 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:18.204409 (+ 696us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:18.204717 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.205756 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.208796 (+ 3040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.211024 (+ 2228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.211518 (+ 494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.211975 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:18.212085 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 01:19:18.212437 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29310,"thread_start_us":29102,"threads_started":1}
I20260806 01:19:18.213143 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.172980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:44185 (local address 127.0.0.1:46564)
0806 01:19:18.186081 (+ 13101us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:18.186109 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:18.186293 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:18.203967 (+ 17674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:18.203972 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:18.203999 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:18.204220 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:18.204234 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.205920 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.205925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:18.208348 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:18.208407 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.211487 (+ 3080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.211496 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:18.211735 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.211782 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 01:19:18.212008 (+ 226us) client_negotiation.cc:240] Negotiation successful
0806 01:19:18.212816 (+ 808us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12917,"thread_start_us":30254,"threads_started":1}
I20260806 01:19:18.214990 25016 tablet_bootstrap.cc:492] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891: Bootstrap starting.
I20260806 01:19:18.215304 25017 tablet_bootstrap.cc:492] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09: Bootstrap starting.
I20260806 01:19:18.217931 24882 tablet_service.cc:1467] Processing CreateTablet for tablet 96052b3c23ef49ecb7dc91201c6e65d1 (DEFAULT_TABLE table=test-table [id=0cc2767a02ee494a9467cea277d79896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:19:18.218839 25016 tablet_bootstrap.cc:654] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:18.219697 24882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96052b3c23ef49ecb7dc91201c6e65d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:18.220329 25016 log.cc:824] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:18.220583 25017 tablet_bootstrap.cc:654] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:18.221764 25017 log.cc:824] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:18.224295 25016 tablet_bootstrap.cc:492] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891: No bootstrap required, opened a new log
I20260806 01:19:18.224604 25016 ts_tablet_manager.cc:1397] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260806 01:19:18.225356 25017 tablet_bootstrap.cc:492] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09: No bootstrap required, opened a new log
I20260806 01:19:18.225672 25017 ts_tablet_manager.cc:1397] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260806 01:19:18.234264 25021 tablet_bootstrap.cc:492] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58: Bootstrap starting.
I20260806 01:19:18.236565 25016 raft_consensus.cc:348] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419ed089854243b48be4ae1b6b0a3891" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35797 } } peers { permanent_uuid: "22cd371b03ec435aadfacf233ff4ff09" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35175 } } peers { permanent_uuid: "61b05cb34f5a4e0ea967f51f6c423c58" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44185 } }
I20260806 01:19:18.237072 25016 raft_consensus.cc:374] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:18.237223 25016 raft_consensus.cc:729] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 419ed089854243b48be4ae1b6b0a3891, State: Initialized, Role: FOLLOWER
I20260806 01:19:18.237466 25021 tablet_bootstrap.cc:654] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:18.237265 25017 raft_consensus.cc:348] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419ed089854243b48be4ae1b6b0a3891" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35797 } } peers { permanent_uuid: "22cd371b03ec435aadfacf233ff4ff09" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35175 } } peers { permanent_uuid: "61b05cb34f5a4e0ea967f51f6c423c58" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44185 } }
I20260806 01:19:18.237777 25017 raft_consensus.cc:374] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:18.237888 25017 raft_consensus.cc:729] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22cd371b03ec435aadfacf233ff4ff09, State: Initialized, Role: FOLLOWER
I20260806 01:19:18.237937 25016 consensus_queue.cc:260] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419ed089854243b48be4ae1b6b0a3891" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35797 } } peers { permanent_uuid: "22cd371b03ec435aadfacf233ff4ff09" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35175 } } peers { permanent_uuid: "61b05cb34f5a4e0ea967f51f6c423c58" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44185 } }
I20260806 01:19:18.238519 25017 consensus_queue.cc:260] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419ed089854243b48be4ae1b6b0a3891" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35797 } } peers { permanent_uuid: "22cd371b03ec435aadfacf233ff4ff09" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35175 } } peers { permanent_uuid: "61b05cb34f5a4e0ea967f51f6c423c58" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44185 } }
I20260806 01:19:18.238914 25021 log.cc:824] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:18.239746 24801 heartbeater.cc:503] Master 127.25.236.190:45947 was elected leader, sending a full tablet report...
I20260806 01:19:18.240442 25017 ts_tablet_manager.cc:1428] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260806 01:19:18.241422 25016 ts_tablet_manager.cc:1428] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260806 01:19:18.243876 25021 tablet_bootstrap.cc:492] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58: No bootstrap required, opened a new log
I20260806 01:19:18.244182 25021 ts_tablet_manager.cc:1397] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 01:19:18.255422 25021 raft_consensus.cc:348] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419ed089854243b48be4ae1b6b0a3891" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35797 } } peers { permanent_uuid: "22cd371b03ec435aadfacf233ff4ff09" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35175 } } peers { permanent_uuid: "61b05cb34f5a4e0ea967f51f6c423c58" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44185 } }
I20260806 01:19:18.255932 25021 raft_consensus.cc:374] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:18.256047 25021 raft_consensus.cc:729] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b05cb34f5a4e0ea967f51f6c423c58, State: Initialized, Role: FOLLOWER
I20260806 01:19:18.256697 25021 consensus_queue.cc:260] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419ed089854243b48be4ae1b6b0a3891" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35797 } } peers { permanent_uuid: "22cd371b03ec435aadfacf233ff4ff09" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35175 } } peers { permanent_uuid: "61b05cb34f5a4e0ea967f51f6c423c58" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44185 } }
I20260806 01:19:18.257833 24980 heartbeater.cc:503] Master 127.25.236.190:45947 was elected leader, sending a full tablet report...
I20260806 01:19:18.258671 25021 ts_tablet_manager.cc:1428] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58: Time spent starting tablet: real 0.014s user 0.011s sys 0.005s
I20260806 01:19:18.282424 25025 raft_consensus.cc:482] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:19:18.282784 25025 raft_consensus.cc:504] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419ed089854243b48be4ae1b6b0a3891" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35797 } } peers { permanent_uuid: "22cd371b03ec435aadfacf233ff4ff09" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35175 } } peers { permanent_uuid: "61b05cb34f5a4e0ea967f51f6c423c58" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44185 } }
I20260806 01:19:18.285020 25025 leader_election.cc:288] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 419ed089854243b48be4ae1b6b0a3891 (127.25.236.129:35797), 61b05cb34f5a4e0ea967f51f6c423c58 (127.25.236.131:44185)
I20260806 01:19:18.295701 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.285541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:34431 (local address 127.25.236.131:44185)
0806 01:19:18.285745 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:18.285753 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:18.286542 (+ 789us) server_negotiation.cc:407] Connection header received
0806 01:19:18.286805 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:18.286810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:18.286894 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:18.286997 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:18.287985 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.289117 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.293109 (+ 3992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.294396 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.294596 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.294665 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:18.295311 (+ 646us) server_negotiation.cc:299] Negotiation successful
0806 01:19:18.295446 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 01:19:18.295711 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.285683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.236.130:43509 (local address 127.25.236.129:35797)
0806 01:19:18.285890 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:18.285897 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:18.286542 (+ 645us) server_negotiation.cc:407] Connection header received
0806 01:19:18.286805 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:18.286810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:18.286895 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:18.286997 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:19:18.287985 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.289115 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.293105 (+ 3990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.294415 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.294575 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.294665 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:18.295311 (+ 646us) server_negotiation.cc:299] Negotiation successful
0806 01:19:18.295448 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:19:18.295922 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.285381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.129:35797 (local address 127.25.236.130:43509)
0806 01:19:18.286408 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:18.286429 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:18.286612 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:18.287029 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:18.287034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:18.287066 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:18.287773 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:18.287788 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.289356 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.289363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:18.292680 (+ 3317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:18.292717 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.294690 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.294698 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:18.294964 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.295055 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 01:19:18.295177 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 01:19:18.295369 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"mutex_wait_us":306,"spinlock_wait_cycles":18048,"thread_start_us":509,"threads_started":1}
I20260806 01:19:18.295976 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.285381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.131:44185 (local address 127.25.236.130:34431)
0806 01:19:18.286413 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0806 01:19:18.286435 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:19:18.286617 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:19:18.287029 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:19:18.287034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:19:18.287069 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:19:18.287773 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:18.287788 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.289357 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.289363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:18.292680 (+ 3317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:19:18.292717 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.294696 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.294702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:19:18.294964 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.295061 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 01:19:18.295176 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 01:19:18.295369 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":923,"mutex_wait_us":530,"spinlock_wait_cycles":38016,"thread_start_us":598,"threads_started":1}
I20260806 01:19:18.297009 24916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96052b3c23ef49ecb7dc91201c6e65d1" candidate_uuid: "22cd371b03ec435aadfacf233ff4ff09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b05cb34f5a4e0ea967f51f6c423c58" is_pre_election: true
I20260806 01:19:18.297008 24555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96052b3c23ef49ecb7dc91201c6e65d1" candidate_uuid: "22cd371b03ec435aadfacf233ff4ff09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "419ed089854243b48be4ae1b6b0a3891" is_pre_election: true
I20260806 01:19:18.297524 24916 raft_consensus.cc:2454] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22cd371b03ec435aadfacf233ff4ff09 in term 0.
I20260806 01:19:18.297523 24555 raft_consensus.cc:2454] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22cd371b03ec435aadfacf233ff4ff09 in term 0.
I20260806 01:19:18.298280 24651 leader_election.cc:302] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 22cd371b03ec435aadfacf233ff4ff09, 419ed089854243b48be4ae1b6b0a3891, 61b05cb34f5a4e0ea967f51f6c423c58; no voters:
I20260806 01:19:18.298672 25025 raft_consensus.cc:2781] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:19:18.298837 25025 raft_consensus.cc:482] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:19:18.298957 25025 raft_consensus.cc:3037] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:18.301209 25025 raft_consensus.cc:504] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419ed089854243b48be4ae1b6b0a3891" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35797 } } peers { permanent_uuid: "22cd371b03ec435aadfacf233ff4ff09" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35175 } } peers { permanent_uuid: "61b05cb34f5a4e0ea967f51f6c423c58" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44185 } }
I20260806 01:19:18.301899 25025 leader_election.cc:288] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [CANDIDATE]: Term 1 election: Requested vote from peers 419ed089854243b48be4ae1b6b0a3891 (127.25.236.129:35797), 61b05cb34f5a4e0ea967f51f6c423c58 (127.25.236.131:44185)
I20260806 01:19:18.302417 24555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96052b3c23ef49ecb7dc91201c6e65d1" candidate_uuid: "22cd371b03ec435aadfacf233ff4ff09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "419ed089854243b48be4ae1b6b0a3891"
I20260806 01:19:18.302569 24916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96052b3c23ef49ecb7dc91201c6e65d1" candidate_uuid: "22cd371b03ec435aadfacf233ff4ff09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b05cb34f5a4e0ea967f51f6c423c58"
I20260806 01:19:18.302644 24555 raft_consensus.cc:3037] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:18.302819 24916 raft_consensus.cc:3037] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:18.305312 24555 raft_consensus.cc:2454] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22cd371b03ec435aadfacf233ff4ff09 in term 1.
I20260806 01:19:18.305673 24916 raft_consensus.cc:2454] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22cd371b03ec435aadfacf233ff4ff09 in term 1.
I20260806 01:19:18.305757 24656 leader_election.cc:302] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22cd371b03ec435aadfacf233ff4ff09, 419ed089854243b48be4ae1b6b0a3891; no voters:
I20260806 01:19:18.306094 25025 raft_consensus.cc:2781] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:18.306839 25025 raft_consensus.cc:686] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [term 1 LEADER]: Becoming Leader. State: Replica: 22cd371b03ec435aadfacf233ff4ff09, State: Running, Role: LEADER
I20260806 01:19:18.307353 25025 consensus_queue.cc:237] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419ed089854243b48be4ae1b6b0a3891" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35797 } } peers { permanent_uuid: "22cd371b03ec435aadfacf233ff4ff09" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35175 } } peers { permanent_uuid: "61b05cb34f5a4e0ea967f51f6c423c58" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44185 } }
I20260806 01:19:18.312337 24351 catalog_manager.cc:5495] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22cd371b03ec435aadfacf233ff4ff09 (127.25.236.130). New cstate: current_term: 1 leader_uuid: "22cd371b03ec435aadfacf233ff4ff09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419ed089854243b48be4ae1b6b0a3891" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22cd371b03ec435aadfacf233ff4ff09" member_type: VOTER last_known_addr { host: "127.25.236.130" port: 35175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61b05cb34f5a4e0ea967f51f6c423c58" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44185 } health_report { overall_health: UNKNOWN } } }
I20260806 01:19:18.340687 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.335334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34272 (local address 127.25.236.130:35175)
0806 01:19:18.335543 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:18.335549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:18.335583 (+ 34us) server_negotiation.cc:407] Connection header received
0806 01:19:18.335690 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:18.335694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:18.335782 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:18.335944 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:19:18.336310 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.337035 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.338552 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.338957 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.339117 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.339594 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:19:18.340101 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:19:18.340272 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:18.340331 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 01:19:18.340411 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 01:19:18.348981 24622 tablet.cc:2369] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:19:18.349510 24916 raft_consensus.cc:1261] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58 [term 1 FOLLOWER]: Refusing update from remote peer 22cd371b03ec435aadfacf233ff4ff09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:19:18.349542 24555 raft_consensus.cc:1261] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891 [term 1 FOLLOWER]: Refusing update from remote peer 22cd371b03ec435aadfacf233ff4ff09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:19:18.350919 25032 consensus_queue.cc:1035] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "419ed089854243b48be4ae1b6b0a3891" member_type: VOTER last_known_addr { host: "127.25.236.129" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:19:18.351305 25025 consensus_queue.cc:1035] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61b05cb34f5a4e0ea967f51f6c423c58" member_type: VOTER last_known_addr { host: "127.25.236.131" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:19:18.356559 24803 tablet.cc:2369] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:19:18.359167 24981 tablet.cc:2369] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:19:18.370182 25036 mvcc.cc:205] Tried to move back new op lower bound from 7315370632588595200 to 7315370632431185920. Current Snapshot: MvccSnapshot[applied={T|T < 7315370632588595200}]
I20260806 01:19:18.378684 24350 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:19:18.379002 24350 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:19:18.382876 24350 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9266564eba9c44ab8334a6baf94718dd: Sending DeleteTablet for 3 replicas of tablet 96052b3c23ef49ecb7dc91201c6e65d1
I20260806 01:19:18.384052 24712 tablet_service.cc:1514] Processing DeleteTablet for tablet 96052b3c23ef49ecb7dc91201c6e65d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:18 UTC) from {username='slave'} at 127.0.0.1:34256
I20260806 01:19:18.384068 24526 tablet_service.cc:1514] Processing DeleteTablet for tablet 96052b3c23ef49ecb7dc91201c6e65d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:18 UTC) from {username='slave'} at 127.0.0.1:39554
I20260806 01:19:18.384599 24882 tablet_service.cc:1514] Processing DeleteTablet for tablet 96052b3c23ef49ecb7dc91201c6e65d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:19:18 UTC) from {username='slave'} at 127.0.0.1:46564
I20260806 01:19:18.385195 25046 tablet_replica.cc:330] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09: stopping tablet replica
I20260806 01:19:18.385604 25046 raft_consensus.cc:2229] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:19:18.386582 25046 raft_consensus.cc:2258] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:18.386075 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 24432
I20260806 01:19:18.388955 25048 tablet_replica.cc:330] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58: stopping tablet replica
I20260806 01:19:18.389408 25048 raft_consensus.cc:2229] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:19:18.389804 25046 ts_tablet_manager.cc:1905] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:18.390048 25048 raft_consensus.cc:2258] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:18.392515 25048 ts_tablet_manager.cc:1905] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:18.393239 25047 tablet_replica.cc:330] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891: stopping tablet replica
I20260806 01:19:18.394707 25047 raft_consensus.cc:2229] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:19:18.395274 25047 raft_consensus.cc:2258] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:19:18.397671 25046 ts_tablet_manager.cc:1918] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:18.397863 25046 log.cc:1196] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-1/wal/wals/96052b3c23ef49ecb7dc91201c6e65d1
I20260806 01:19:18.398561 25046 ts_tablet_manager.cc:1939] T 96052b3c23ef49ecb7dc91201c6e65d1 P 22cd371b03ec435aadfacf233ff4ff09: Deleting consensus metadata
I20260806 01:19:18.400660 25047 ts_tablet_manager.cc:1905] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:19:18.400723 24328 catalog_manager.cc:4841] TS 22cd371b03ec435aadfacf233ff4ff09 (127.25.236.130:35175): tablet 96052b3c23ef49ecb7dc91201c6e65d1 (table test-table [id=0cc2767a02ee494a9467cea277d79896]) successfully deleted
I20260806 01:19:18.405560 25048 ts_tablet_manager.cc:1918] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:18.405764 25048 log.cc:1196] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-2/wal/wals/96052b3c23ef49ecb7dc91201c6e65d1
I20260806 01:19:18.406370 25048 ts_tablet_manager.cc:1939] T 96052b3c23ef49ecb7dc91201c6e65d1 P 61b05cb34f5a4e0ea967f51f6c423c58: Deleting consensus metadata
I20260806 01:19:18.408293 24326 catalog_manager.cc:4841] TS 61b05cb34f5a4e0ea967f51f6c423c58 (127.25.236.131:44185): tablet 96052b3c23ef49ecb7dc91201c6e65d1 (table test-table [id=0cc2767a02ee494a9467cea277d79896]) successfully deleted
I20260806 01:19:18.409180 25047 ts_tablet_manager.cc:1918] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:19:18.409389 25047 log.cc:1196] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891: Deleting WAL directory at /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978805316623-26546-0/minicluster-data/ts-0/wal/wals/96052b3c23ef49ecb7dc91201c6e65d1
I20260806 01:19:18.409966 25047 ts_tablet_manager.cc:1939] T 96052b3c23ef49ecb7dc91201c6e65d1 P 419ed089854243b48be4ae1b6b0a3891: Deleting consensus metadata
I20260806 01:19:18.411803 24331 catalog_manager.cc:4841] TS 419ed089854243b48be4ae1b6b0a3891 (127.25.236.129:35797): tablet 96052b3c23ef49ecb7dc91201c6e65d1 (table test-table [id=0cc2767a02ee494a9467cea277d79896]) successfully deleted
I20260806 01:19:18.414979 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.388370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39566 (local address 127.25.236.129:35797)
0806 01:19:18.388559 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:18.388566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:18.406838 (+ 18272us) server_negotiation.cc:407] Connection header received
0806 01:19:18.407081 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:18.407087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:18.407202 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:18.407285 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:18.407988 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.409140 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.411190 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.411898 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.412030 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.413155 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:18.413167 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:18.413186 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:18.414000 (+ 814us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:18.414060 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:18.414086 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:18.414210 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:18.414474 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 01:19:18.414600 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:19:18.415906 24614 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:18.517956 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 24432
I20260806 01:19:18.554682 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 24625
I20260806 01:19:18.570849 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.555778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34274 (local address 127.25.236.130:35175)
0806 01:19:18.556014 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:18.556021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:18.556227 (+ 206us) server_negotiation.cc:407] Connection header received
0806 01:19:18.556385 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:18.556391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:18.556497 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:18.556625 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:18.557053 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.559183 (+ 2130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.560576 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.561314 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.561457 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.568849 (+ 7392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:18.568863 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:18.568886 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:18.569779 (+ 893us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:18.569846 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:18.569874 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:18.570009 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:18.570205 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 01:19:18.570363 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 01:19:18.571789 24794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:18.696532 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 24625
I20260806 01:19:18.727011 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 24805
I20260806 01:19:18.734081 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:18.727732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46576 (local address 127.25.236.131:44185)
0806 01:19:18.727922 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:18.727929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:18.728004 (+ 75us) server_negotiation.cc:407] Connection header received
0806 01:19:18.728227 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:18.728231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:18.728322 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:18.728432 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:18.728853 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.729581 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.730926 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:18.731311 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:18.731444 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:18.732583 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:18.732593 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:18.732608 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:18.733307 (+ 699us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:18.733359 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:18.733388 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:18.733528 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:18.733672 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 01:19:18.733779 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:19:18.734825 24972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:18.828380 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 24805
I20260806 01:19:18.861366 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 24304
I20260806 01:19:18.862952 24394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:18.973286 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 24304
2026-08-06T01:19:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3108 ms)
[----------] 6 tests from EncryptionPolicyTest (26082 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:19 dist-test-slave-7nm7 krb5kdc[25059](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:19 dist-test-slave-7nm7 krb5kdc[25059](info): set up 2 sockets
Aug 06 01:19:19 dist-test-slave-7nm7 krb5kdc[25059](info): commencing operation
krb5kdc: starting...
W20260806 01:19:20.346199 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:20 dist-test-slave-7nm7 krb5kdc[25059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:19:20.717471 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34147
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:21.095384 25078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:21.095700 25078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:21.095880 25078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:21.106092 25078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:21.106254 25078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:21.106305 25078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:21.106441 25078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:21.106508 25078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:21.119850 25078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34147
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:21.121467 25078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:21.123062 25078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:21.136310 25085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:21.136565 25088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:21.137305 25090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:21.139981 25078 server_base.cc:1034] running on GCE node
I20260806 01:19:21.141645 25078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:21.143841 25078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:21.145147 25078 hybrid_clock.cc:648] HybridClock initialized: now 1785979161145039 us; error 126 us; skew 500 ppm
I20260806 01:19:21.145555 25078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:21 dist-test-slave-7nm7 krb5kdc[25059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979161, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:21.149062 25078 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:19:21.150842 25078 webserver.cc:460] Webserver started at http://127.0.0.1:41125/ using document root <none> and password file <none>
I20260806 01:19:21.151599 25078 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:21.151758 25078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:21.152089 25078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:21.155359 25078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "82b734fcd07145749e03255d0a84f00d"
format_stamp: "Formatted at 2026-08-06 01:19:21 on dist-test-slave-7nm7"
server_key: "1769fa84349b0cd8229d2ce874f104f9"
server_key_iv: "9d26765c1e61fb60630e0d9fa90a0418"
server_key_version: "encryptionkey@0"
I20260806 01:19:21.156188 25078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "82b734fcd07145749e03255d0a84f00d"
format_stamp: "Formatted at 2026-08-06 01:19:21 on dist-test-slave-7nm7"
server_key: "1769fa84349b0cd8229d2ce874f104f9"
server_key_iv: "9d26765c1e61fb60630e0d9fa90a0418"
server_key_version: "encryptionkey@0"
I20260806 01:19:21.162568 25078 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 01:19:21.166421 25100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:21.167879 25078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:19:21.168083 25078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "82b734fcd07145749e03255d0a84f00d"
format_stamp: "Formatted at 2026-08-06 01:19:21 on dist-test-slave-7nm7"
server_key: "1769fa84349b0cd8229d2ce874f104f9"
server_key_iv: "9d26765c1e61fb60630e0d9fa90a0418"
server_key_version: "encryptionkey@0"
I20260806 01:19:21.168293 25078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:21.202283 25078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:21.207021 25078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:21.207352 25078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:21.228241 25078 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34147
I20260806 01:19:21.228295 25171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34147 every 8 connection(s)
I20260806 01:19:21.229681 25078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:21.233381 25174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:21.236750 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 25078
I20260806 01:19:21.237130 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:21.237846 26546 external_mini_cluster.cc:1428] Setting key 3d43d0ae1eb126f208b706c25edb2ed3
I20260806 01:19:21.247380 25174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d: Bootstrap starting.
I20260806 01:19:21.250579 25174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:21.251593 25174 log.cc:824] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d: Log is configured to *not* fsync() on all Append() calls
Aug 06 01:19:21 dist-test-slave-7nm7 krb5kdc[25059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 01:19:21.255543 25174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d: No bootstrap required, opened a new log
I20260806 01:19:21.262971 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:21.240002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55169 (local address 127.0.0.1:34147)
0806 01:19:21.241056 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:21.241068 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:21.241235 (+ 167us) server_negotiation.cc:407] Connection header received
0806 01:19:21.243343 (+ 2108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:21.243367 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:21.243769 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:21.244195 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:21.245701 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:21.247105 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:21.248499 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:21.249207 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:21.249402 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:21.254685 (+ 5283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:21.254694 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:21.254708 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:21.254745 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:21.258345 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:21.259282 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:21.259289 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:21.259295 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:21.259468 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:21.260011 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:21.260019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:21.260021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:21.260575 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:21.260808 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:21.261423 (+ 615us) server_negotiation.cc:299] Negotiation successful
0806 01:19:21.261790 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":414,"threads_started":1}
I20260806 01:19:21.266129 25174 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b734fcd07145749e03255d0a84f00d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34147 } }
I20260806 01:19:21.266669 25174 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:21.266785 25174 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b734fcd07145749e03255d0a84f00d, State: Initialized, Role: FOLLOWER
I20260806 01:19:21.267481 25174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b734fcd07145749e03255d0a84f00d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34147 } }
I20260806 01:19:21.267673 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:21.267771 25174 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:21.267925 25174 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:21.268105 25174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:21.270542 25174 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b734fcd07145749e03255d0a84f00d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34147 } }
I20260806 01:19:21.271095 25174 leader_election.cc:302] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82b734fcd07145749e03255d0a84f00d; no voters:
I20260806 01:19:21.271811 25174 leader_election.cc:288] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:21.271965 25189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:21.273470 25189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [term 1 LEADER]: Becoming Leader. State: Replica: 82b734fcd07145749e03255d0a84f00d, State: Running, Role: LEADER
I20260806 01:19:21.274739 25174 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:21.275027 25189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b734fcd07145749e03255d0a84f00d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34147 } }
I20260806 01:19:21.278077 25192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82b734fcd07145749e03255d0a84f00d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b734fcd07145749e03255d0a84f00d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34147 } } }
I20260806 01:19:21.278493 25192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:21.278105 25193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82b734fcd07145749e03255d0a84f00d. Latest consensus state: current_term: 1 leader_uuid: "82b734fcd07145749e03255d0a84f00d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b734fcd07145749e03255d0a84f00d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34147 } } }
I20260806 01:19:21.279485 25193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d [sys.catalog]: This master's current role is: LEADER
W20260806 01:19:21.293064 25215 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:19:21.293211 25215 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:19:21.293376 25200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:21.295291 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:21.274045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50821 (local address 127.0.0.1:34147)
0806 01:19:21.274330 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:21.274339 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:21.274750 (+ 411us) server_negotiation.cc:407] Connection header received
0806 01:19:21.274978 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:21.274984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:21.275076 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:21.275201 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:21.277688 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:21.279524 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:21.282108 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:21.283089 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:21.283321 (+ 232us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:21.287795 (+ 4474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:21.287803 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:21.287812 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:21.287868 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:21.291533 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:21.292538 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:21.292544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:21.292547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:21.292726 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:21.293415 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:21.293424 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:21.293427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:21.293888 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:21.294185 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:21.294738 (+ 553us) server_negotiation.cc:299] Negotiation successful
0806 01:19:21.294918 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:19:21.295758 25200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:21.305198 25200 catalog_manager.cc:1347] Generated new cluster ID: ca2181e4a4174e82a22a53b70a23276d
I20260806 01:19:21.305338 25200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:21.321655 25200 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:21.323037 25200 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:21.330718 25200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82b734fcd07145749e03255d0a84f00d: Generated new TSK 0
I20260806 01:19:21.331749 25200 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:21.341535 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 25078
I20260806 01:19:21.343011 25163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:21.438903 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 25078
W20260806 01:19:21.466172 25057 connection.cc:535] client connection to 127.0.0.1:34147 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34147): Connection reset by peer (error 104)
2026-08-06T01:19:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:21 dist-test-slave-7nm7 krb5kdc[25223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:21 dist-test-slave-7nm7 krb5kdc[25223](info): set up 2 sockets
Aug 06 01:19:21 dist-test-slave-7nm7 krb5kdc[25223](info): commencing operation
krb5kdc: starting...
W20260806 01:19:22.841722 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.356s user 0.007s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:23 dist-test-slave-7nm7 krb5kdc[25223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:19:23.205757 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42067
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:23.587852 25241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:23.588174 25241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:23.588339 25241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:23.597872 25241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:23.598352 25241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:23.598461 25241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:23.598601 25241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:23.598706 25241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:23.611223 25241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42067
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:23.612722 25241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:23.614197 25241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:23.630829 25251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:23.630720 25248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:23.630791 25253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:23.632826 25241 server_base.cc:1034] running on GCE node
I20260806 01:19:23.633783 25241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:23.636229 25241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:23.637614 25241 hybrid_clock.cc:648] HybridClock initialized: now 1785979163637538 us; error 72 us; skew 500 ppm
I20260806 01:19:23.638057 25241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:23 dist-test-slave-7nm7 krb5kdc[25223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979163, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:23.646142 25241 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:19:23.648025 25241 webserver.cc:460] Webserver started at http://127.0.0.1:33005/ using document root <none> and password file <none>
I20260806 01:19:23.648802 25241 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:23.648943 25241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:23.649290 25241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:23.652513 25241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "e5940e7433ab448daaddfa2624fa8df2"
format_stamp: "Formatted at 2026-08-06 01:19:23 on dist-test-slave-7nm7"
server_key: "bee1ca09e32869fb2ce695aa6373f3d9"
server_key_iv: "0fb1a925539f32bf87a641a9ffc27ce0"
server_key_version: "encryptionkey@0"
I20260806 01:19:23.653520 25241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "e5940e7433ab448daaddfa2624fa8df2"
format_stamp: "Formatted at 2026-08-06 01:19:23 on dist-test-slave-7nm7"
server_key: "bee1ca09e32869fb2ce695aa6373f3d9"
server_key_iv: "0fb1a925539f32bf87a641a9ffc27ce0"
server_key_version: "encryptionkey@0"
I20260806 01:19:23.660472 25241 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.008s
I20260806 01:19:23.664834 25262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:23.666392 25241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:19:23.666611 25241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "e5940e7433ab448daaddfa2624fa8df2"
format_stamp: "Formatted at 2026-08-06 01:19:23 on dist-test-slave-7nm7"
server_key: "bee1ca09e32869fb2ce695aa6373f3d9"
server_key_iv: "0fb1a925539f32bf87a641a9ffc27ce0"
server_key_version: "encryptionkey@0"
I20260806 01:19:23.666810 25241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:23.691529 25241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:23.697316 25241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:23.697682 25241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:23.720948 25241 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42067
I20260806 01:19:23.720985 25342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42067 every 8 connection(s)
I20260806 01:19:23.722378 25241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:23.725541 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 25241
I20260806 01:19:23.725919 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:23.726336 25343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:23.726627 26546 external_mini_cluster.cc:1428] Setting key 94cbe023c90243d106ccbf804959d9f3
I20260806 01:19:23.740760 25343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2: Bootstrap starting.
I20260806 01:19:23.741446 25346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:23.728704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50175 (local address 127.0.0.1:42067)
0806 01:19:23.729723 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:23.729735 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:23.729778 (+ 43us) server_negotiation.cc:407] Connection header received
0806 01:19:23.731420 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:23.731441 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:23.731850 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:23.732218 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:23.732847 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:23.734225 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:23.736091 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:23.736840 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:23.737061 (+ 221us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:23.738448 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:23.738463 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:23.738478 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:23.739202 (+ 724us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:23.739279 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:23.739307 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:23.739440 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:23.739862 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 01:19:23.740080 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":792,"thread_start_us":558,"threads_started":1}
I20260806 01:19:23.745438 25343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:23.746347 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:23.746876 25343 log.cc:824] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:23.750145 25343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2: No bootstrap required, opened a new log
I20260806 01:19:23.758997 25346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:23.751341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50013 (local address 127.0.0.1:42067)
0806 01:19:23.751553 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:23.751563 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:23.752180 (+ 617us) server_negotiation.cc:407] Connection header received
0806 01:19:23.752294 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:23.752298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:23.752369 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:23.752455 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:23.753128 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:23.753969 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:23.756099 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:23.756644 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:23.756767 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:23.758077 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:23.758082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:23.758087 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:23.758116 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:23.758158 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:23.758178 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:23.758378 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:23.758576 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 01:19:23.758624 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:19:23.761273 25343 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5940e7433ab448daaddfa2624fa8df2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42067 } }
I20260806 01:19:23.762143 25343 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:23.762382 25343 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5940e7433ab448daaddfa2624fa8df2, State: Initialized, Role: FOLLOWER
I20260806 01:19:23.763067 25343 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5940e7433ab448daaddfa2624fa8df2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42067 } }
I20260806 01:19:23.763392 25343 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:23.763586 25343 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:23.763823 25343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:23.766645 25343 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5940e7433ab448daaddfa2624fa8df2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42067 } }
I20260806 01:19:23.767242 25343 leader_election.cc:302] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5940e7433ab448daaddfa2624fa8df2; no voters:
I20260806 01:19:23.767868 25343 leader_election.cc:288] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:23.768036 25356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:23.769219 25356 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [term 1 LEADER]: Becoming Leader. State: Replica: e5940e7433ab448daaddfa2624fa8df2, State: Running, Role: LEADER
I20260806 01:19:23.770026 25356 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5940e7433ab448daaddfa2624fa8df2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42067 } }
I20260806 01:19:23.770401 25343 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:23.774976 25357 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5940e7433ab448daaddfa2624fa8df2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5940e7433ab448daaddfa2624fa8df2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42067 } } }
I20260806 01:19:23.774952 25358 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5940e7433ab448daaddfa2624fa8df2. Latest consensus state: current_term: 1 leader_uuid: "e5940e7433ab448daaddfa2624fa8df2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5940e7433ab448daaddfa2624fa8df2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42067 } } }
I20260806 01:19:23.775436 25357 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:23.775444 25358 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:23.780416 25368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:23.785207 25368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:23.795847 25368 catalog_manager.cc:1347] Generated new cluster ID: 1f21a06941724858a9ddf222ecf452cd
I20260806 01:19:23.795995 25368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:23.809010 25368 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:23.810508 25368 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:23.822081 25368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5940e7433ab448daaddfa2624fa8df2: Generated new TSK 0
I20260806 01:19:23.824139 25368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:23.838341 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 25241
W20260806 01:19:23.840360 25333 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50175
W20260806 01:19:23.841256 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 25241: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:19:23.841466 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 25241
W20260806 01:19:23.871289 25219 connection.cc:535] client connection to 127.0.0.1:42067 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42067): Connection reset by peer (error 104)
2026-08-06T01:19:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:23 dist-test-slave-7nm7 krb5kdc[25382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:23 dist-test-slave-7nm7 krb5kdc[25382](info): set up 2 sockets
Aug 06 01:19:23 dist-test-slave-7nm7 krb5kdc[25382](info): commencing operation
krb5kdc: starting...
W20260806 01:19:25.270525 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.379s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:25 dist-test-slave-7nm7 krb5kdc[25382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:19:25.660653 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35389
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:26.052450 25400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:26.052845 25400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:26.053074 25400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:26.061471 25400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:26.061672 25400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:26.061792 25400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:26.061901 25400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:26.061986 25400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:26.073262 25400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35389
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:26.074816 25400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:26.076174 25400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:26.089876 25410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:26.090077 25409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:26.089913 25417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:26.091184 25400 server_base.cc:1034] running on GCE node
I20260806 01:19:26.092027 25400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:26.093564 25400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:26.094811 25400 hybrid_clock.cc:648] HybridClock initialized: now 1785979166094751 us; error 50 us; skew 500 ppm
I20260806 01:19:26.095187 25400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:26 dist-test-slave-7nm7 krb5kdc[25382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979166, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:26.098619 25400 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:19:26.100179 25400 webserver.cc:460] Webserver started at http://127.0.0.1:45853/ using document root <none> and password file <none>
I20260806 01:19:26.100871 25400 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:26.101027 25400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:26.101366 25400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:26.104468 25400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "f475a04a648340bda7bba913c844cb8a"
format_stamp: "Formatted at 2026-08-06 01:19:26 on dist-test-slave-7nm7"
server_key: "fcc38939a44558abd290a59193cb2638"
server_key_iv: "94e942389cccd00e0fda888b581b18ae"
server_key_version: "encryptionkey@0"
I20260806 01:19:26.105268 25400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "f475a04a648340bda7bba913c844cb8a"
format_stamp: "Formatted at 2026-08-06 01:19:26 on dist-test-slave-7nm7"
server_key: "fcc38939a44558abd290a59193cb2638"
server_key_iv: "94e942389cccd00e0fda888b581b18ae"
server_key_version: "encryptionkey@0"
I20260806 01:19:26.110467 25400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 01:19:26.113965 25424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:26.115432 25400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:26.115604 25400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "f475a04a648340bda7bba913c844cb8a"
format_stamp: "Formatted at 2026-08-06 01:19:26 on dist-test-slave-7nm7"
server_key: "fcc38939a44558abd290a59193cb2638"
server_key_iv: "94e942389cccd00e0fda888b581b18ae"
server_key_version: "encryptionkey@0"
I20260806 01:19:26.115811 25400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:26.121668 25400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:26.126049 25400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:26.126442 25400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:26.146872 25400 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35389
I20260806 01:19:26.146925 25490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35389 every 8 connection(s)
I20260806 01:19:26.148365 25400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:26.152426 25491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:26.158674 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 25400
I20260806 01:19:26.158888 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:26.159471 26546 external_mini_cluster.cc:1428] Setting key d6e9a3138e6f7281f8ba8fbbb9e10c12
I20260806 01:19:26.167256 25491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a: Bootstrap starting.
I20260806 01:19:26.170706 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:26.162552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53291 (local address 127.0.0.1:35389)
0806 01:19:26.164051 (+ 1499us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:26.164076 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:26.164150 (+ 74us) server_negotiation.cc:407] Connection header received
0806 01:19:26.166583 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:26.166615 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:26.167085 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:26.167594 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:26.168629 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:26.168637 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:26.168655 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:26.168674 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:26.168745 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:26.168778 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:26.169021 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:26.169201 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 01:19:26.169418 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1194,"thread_start_us":768,"threads_started":1}
I20260806 01:19:26.171505 25491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:26.172760 25491 log.cc:824] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:26.176261 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:26.176429 25491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a: No bootstrap required, opened a new log
I20260806 01:19:26.185340 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:26.182473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39603 (local address 127.0.0.1:35389)
0806 01:19:26.182696 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:26.182702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:26.183032 (+ 330us) server_negotiation.cc:407] Connection header received
0806 01:19:26.183227 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:26.183231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:26.183293 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:26.183392 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:26.184413 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:26.184417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:26.184429 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:26.184446 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:26.184497 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:26.184513 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:26.184638 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:26.184928 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 01:19:26.184990 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 01:19:26.185434 25491 raft_consensus.cc:348] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f475a04a648340bda7bba913c844cb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35389 } }
I20260806 01:19:26.185957 25491 raft_consensus.cc:374] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:26.186101 25491 raft_consensus.cc:729] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f475a04a648340bda7bba913c844cb8a, State: Initialized, Role: FOLLOWER
I20260806 01:19:26.186889 25491 consensus_queue.cc:260] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f475a04a648340bda7bba913c844cb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35389 } }
I20260806 01:19:26.187228 25491 raft_consensus.cc:388] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:26.187383 25491 raft_consensus.cc:482] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:26.187520 25491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:26.190376 25491 raft_consensus.cc:504] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f475a04a648340bda7bba913c844cb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35389 } }
I20260806 01:19:26.191059 25491 leader_election.cc:302] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f475a04a648340bda7bba913c844cb8a; no voters:
I20260806 01:19:26.192528 25491 leader_election.cc:288] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:26.192672 25509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:26.193482 25509 raft_consensus.cc:686] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [term 1 LEADER]: Becoming Leader. State: Replica: f475a04a648340bda7bba913c844cb8a, State: Running, Role: LEADER
I20260806 01:19:26.194480 25509 consensus_queue.cc:237] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f475a04a648340bda7bba913c844cb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35389 } }
I20260806 01:19:26.196000 25491 sys_catalog.cc:564] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:26.198338 25510 sys_catalog.cc:455] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f475a04a648340bda7bba913c844cb8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f475a04a648340bda7bba913c844cb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35389 } } }
I20260806 01:19:26.199074 25510 sys_catalog.cc:458] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:26.203459 25512 sys_catalog.cc:455] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f475a04a648340bda7bba913c844cb8a. Latest consensus state: current_term: 1 leader_uuid: "f475a04a648340bda7bba913c844cb8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f475a04a648340bda7bba913c844cb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35389 } } }
I20260806 01:19:26.205243 25512 sys_catalog.cc:458] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:26.210028 25519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:26.216820 25519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:26.228574 25519 catalog_manager.cc:1347] Generated new cluster ID: f067bc72537742348b92ece8cc9088bf
I20260806 01:19:26.228785 25519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:26.245466 25519 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:26.245899 25519 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:26.261456 25519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f475a04a648340bda7bba913c844cb8a: Generated new TSK 0
I20260806 01:19:26.262780 25519 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:26.270507 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 25400
W20260806 01:19:26.271710 25485 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53291
W20260806 01:19:26.272444 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 25400: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:19:26.272588 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 25400
2026-08-06T01:19:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:26 dist-test-slave-7nm7 krb5kdc[25538](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:26 dist-test-slave-7nm7 krb5kdc[25538](info): set up 2 sockets
Aug 06 01:19:26 dist-test-slave-7nm7 krb5kdc[25538](info): commencing operation
krb5kdc: starting...
W20260806 01:19:27.666612 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.345s user 0.002s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:27 dist-test-slave-7nm7 krb5kdc[25538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:19:28.055454 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34523
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:28.438448 25558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:28.438987 25558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:28.439163 25558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:28.449220 25558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:28.449337 25558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:28.449425 25558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:28.449515 25558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:28.449590 25558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:28.461577 25558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:28.463146 25558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:28.464490 25558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:28.477559 25563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:28.477504 25567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:28.477519 25564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:28.478448 25558 server_base.cc:1034] running on GCE node
I20260806 01:19:28.479278 25558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:28.480916 25558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:28.482705 25558 hybrid_clock.cc:648] HybridClock initialized: now 1785979168482597 us; error 41 us; skew 500 ppm
I20260806 01:19:28.483364 25558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:28 dist-test-slave-7nm7 krb5kdc[25538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979168, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:28.486887 25558 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:19:28.488571 25558 webserver.cc:460] Webserver started at http://127.0.0.1:39375/ using document root <none> and password file <none>
I20260806 01:19:28.489283 25558 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:28.489444 25558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:28.489763 25558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:28.493028 25558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "eec523d6999c4d029f530dc76cf138e3"
format_stamp: "Formatted at 2026-08-06 01:19:28 on dist-test-slave-7nm7"
server_key: "fd07a09bb22b9af11010c1dcfcf2a278"
server_key_iv: "8dd5a4db62f2aa010649e3b9ef6bd3f4"
server_key_version: "encryptionkey@0"
I20260806 01:19:28.494171 25558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "eec523d6999c4d029f530dc76cf138e3"
format_stamp: "Formatted at 2026-08-06 01:19:28 on dist-test-slave-7nm7"
server_key: "fd07a09bb22b9af11010c1dcfcf2a278"
server_key_iv: "8dd5a4db62f2aa010649e3b9ef6bd3f4"
server_key_version: "encryptionkey@0"
I20260806 01:19:28.500013 25558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 01:19:28.504204 25583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:28.505746 25558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:19:28.505986 25558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "eec523d6999c4d029f530dc76cf138e3"
format_stamp: "Formatted at 2026-08-06 01:19:28 on dist-test-slave-7nm7"
server_key: "fd07a09bb22b9af11010c1dcfcf2a278"
server_key_iv: "8dd5a4db62f2aa010649e3b9ef6bd3f4"
server_key_version: "encryptionkey@0"
I20260806 01:19:28.506335 25558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:28.523259 25558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:28.528261 25558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:28.528529 25558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:28.550133 25558 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34523
I20260806 01:19:28.550176 25659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34523 every 8 connection(s)
I20260806 01:19:28.551614 25558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:28.553129 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 25558
I20260806 01:19:28.553309 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:28.553884 26546 external_mini_cluster.cc:1428] Setting key d72d8ab19801b0db3a3aebf6d6d88852
I20260806 01:19:28.556121 25662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:19:28 dist-test-slave-7nm7 krb5kdc[25538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 01:19:28.571957 25662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3: Bootstrap starting.
I20260806 01:19:28.577126 25662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:28.578606 25662 log.cc:824] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:28.582193 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:28.556595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47691 (local address 127.0.0.1:34523)
0806 01:19:28.557460 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:28.557471 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:28.557511 (+ 40us) server_negotiation.cc:407] Connection header received
0806 01:19:28.559339 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:28.559363 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:28.559753 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:28.560087 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:28.562392 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:28.564057 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:28.565902 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:28.566707 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:28.566846 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:28.571847 (+ 5001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:28.571858 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:28.571877 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:28.571921 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:28.576700 (+ 4779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:28.577715 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:28.577724 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:28.577733 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:28.577970 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:28.578439 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:28.578445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:28.578448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:28.579497 (+ 1049us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:28.579820 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:28.580367 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 01:19:28.580859 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":409,"threads_started":1}
I20260806 01:19:28.582609 25662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3: No bootstrap required, opened a new log
I20260806 01:19:28.586113 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:28.590785 25662 raft_consensus.cc:348] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec523d6999c4d029f530dc76cf138e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34523 } }
I20260806 01:19:28.591184 25662 raft_consensus.cc:374] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:28.591310 25662 raft_consensus.cc:729] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec523d6999c4d029f530dc76cf138e3, State: Initialized, Role: FOLLOWER
I20260806 01:19:28.591928 25662 consensus_queue.cc:260] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec523d6999c4d029f530dc76cf138e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34523 } }
I20260806 01:19:28.592175 25662 raft_consensus.cc:388] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:28.592326 25662 raft_consensus.cc:482] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:28.592517 25662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:28.595067 25662 raft_consensus.cc:504] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec523d6999c4d029f530dc76cf138e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34523 } }
I20260806 01:19:28.595629 25662 leader_election.cc:302] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eec523d6999c4d029f530dc76cf138e3; no voters:
I20260806 01:19:28.596319 25662 leader_election.cc:288] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:28.596491 25675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:28.597365 25675 raft_consensus.cc:686] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [term 1 LEADER]: Becoming Leader. State: Replica: eec523d6999c4d029f530dc76cf138e3, State: Running, Role: LEADER
I20260806 01:19:28.598093 25675 consensus_queue.cc:237] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec523d6999c4d029f530dc76cf138e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34523 } }
I20260806 01:19:28.600386 25662 sys_catalog.cc:564] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:28.601110 25677 sys_catalog.cc:455] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eec523d6999c4d029f530dc76cf138e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec523d6999c4d029f530dc76cf138e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34523 } } }
I20260806 01:19:28.601508 25677 sys_catalog.cc:458] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:28.602511 25676 sys_catalog.cc:455] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eec523d6999c4d029f530dc76cf138e3. Latest consensus state: current_term: 1 leader_uuid: "eec523d6999c4d029f530dc76cf138e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec523d6999c4d029f530dc76cf138e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34523 } } }
I20260806 01:19:28.602813 25676 sys_catalog.cc:458] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:28.609300 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:28.590579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36727 (local address 127.0.0.1:34523)
0806 01:19:28.590775 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:28.590782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:28.591313 (+ 531us) server_negotiation.cc:407] Connection header received
0806 01:19:28.591550 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:28.591554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:28.591613 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:28.591735 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:28.593377 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:28.594174 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:28.595493 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:28.596102 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:28.596244 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:28.599377 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:28.599383 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:28.599388 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:28.599433 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:28.603763 (+ 4330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:28.605027 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:28.605035 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:28.605041 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:28.605268 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:28.605861 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:28.605869 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:28.605874 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:28.606417 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:28.608222 (+ 1805us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:28.608437 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 01:19:28.608761 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:19:28.610342 25680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:28.616117 25680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:28.625036 25680 catalog_manager.cc:1347] Generated new cluster ID: 4cfe838879aa48148ec9750e48724c1a
I20260806 01:19:28.625175 25680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:28.640741 25680 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:28.642755 25680 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:28.650696 25680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eec523d6999c4d029f530dc76cf138e3: Generated new TSK 0
I20260806 01:19:28.651919 25680 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:28.664279 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 25558
I20260806 01:19:28.665683 25652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:28.760667 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 25558
2026-08-06T01:19:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:28 dist-test-slave-7nm7 krb5kdc[25706](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:28 dist-test-slave-7nm7 krb5kdc[25706](info): set up 2 sockets
Aug 06 01:19:28 dist-test-slave-7nm7 krb5kdc[25706](info): commencing operation
krb5kdc: starting...
W20260806 01:19:30.163647 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.352s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:30 dist-test-slave-7nm7 krb5kdc[25706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:19:30.534977 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37223
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:30.928491 25724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:30.928754 25724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:30.928944 25724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:30.936273 25724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:30.936381 25724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:30.936494 25724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:30.936592 25724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:30.936668 25724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:30.946367 25724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:30.947618 25724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:30.948920 25724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:30.960840 25735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:30.960878 25739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:30.960820 25734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:30.962153 25724 server_base.cc:1034] running on GCE node
I20260806 01:19:30.962952 25724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:30.964331 25724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:30.965569 25724 hybrid_clock.cc:648] HybridClock initialized: now 1785979170965529 us; error 55 us; skew 500 ppm
I20260806 01:19:30.965914 25724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:30 dist-test-slave-7nm7 krb5kdc[25706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979170, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:30.969048 25724 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:19:30.970605 25724 webserver.cc:460] Webserver started at http://127.0.0.1:44603/ using document root <none> and password file <none>
I20260806 01:19:30.971261 25724 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:30.971405 25724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:30.971709 25724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:30.974748 25724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "b70c10a3676643ddb5a9c471ec6833fb"
format_stamp: "Formatted at 2026-08-06 01:19:30 on dist-test-slave-7nm7"
server_key: "320e55c7eb4a0ee589e2c8a0e03ea60d"
server_key_iv: "225accdef4820d47662a95093cbb7877"
server_key_version: "encryptionkey@0"
I20260806 01:19:30.975536 25724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "b70c10a3676643ddb5a9c471ec6833fb"
format_stamp: "Formatted at 2026-08-06 01:19:30 on dist-test-slave-7nm7"
server_key: "320e55c7eb4a0ee589e2c8a0e03ea60d"
server_key_iv: "225accdef4820d47662a95093cbb7877"
server_key_version: "encryptionkey@0"
I20260806 01:19:30.980665 25724 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:19:30.984333 25747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:30.985704 25724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 01:19:30.986001 25724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "b70c10a3676643ddb5a9c471ec6833fb"
format_stamp: "Formatted at 2026-08-06 01:19:30 on dist-test-slave-7nm7"
server_key: "320e55c7eb4a0ee589e2c8a0e03ea60d"
server_key_iv: "225accdef4820d47662a95093cbb7877"
server_key_version: "encryptionkey@0"
I20260806 01:19:30.986212 25724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:31.022425 25724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:31.027482 25724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:31.027844 25724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:31.049340 25724 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37223
I20260806 01:19:31.049536 25821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37223 every 8 connection(s)
I20260806 01:19:31.051028 25724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:31.057201 25822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:31.059679 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 25724
I20260806 01:19:31.059886 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:31.060470 26546 external_mini_cluster.cc:1428] Setting key 18247fedc16024cfa3c8e28aca148c27
I20260806 01:19:31.070463 25822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb: Bootstrap starting.
I20260806 01:19:31.074492 25822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:31.075367 25825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:31.062616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57371 (local address 127.0.0.1:37223)
0806 01:19:31.063514 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:31.063527 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:31.063572 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:19:31.065311 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:31.065333 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:31.065731 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:31.066075 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:31.066798 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:31.068404 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:31.070330 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:31.071275 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:31.071436 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:31.072732 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:31.072740 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:31.072754 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:31.073337 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:31.073419 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:31.073449 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:31.073541 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:31.073801 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 01:19:31.074091 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":424,"threads_started":1}
I20260806 01:19:31.075680 25822 log.cc:824] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:31.079157 25822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb: No bootstrap required, opened a new log
I20260806 01:19:31.079895 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:31.090845 25822 raft_consensus.cc:348] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b70c10a3676643ddb5a9c471ec6833fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37223 } }
I20260806 01:19:31.091497 25822 raft_consensus.cc:374] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:31.091620 25822 raft_consensus.cc:729] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b70c10a3676643ddb5a9c471ec6833fb, State: Initialized, Role: FOLLOWER
I20260806 01:19:31.092265 25822 consensus_queue.cc:260] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b70c10a3676643ddb5a9c471ec6833fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37223 } }
I20260806 01:19:31.092641 25822 raft_consensus.cc:388] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:31.092834 25822 raft_consensus.cc:482] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:31.093065 25822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:31.093441 25825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:31.086073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33705 (local address 127.0.0.1:37223)
0806 01:19:31.086322 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:31.086329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:31.086764 (+ 435us) server_negotiation.cc:407] Connection header received
0806 01:19:31.086920 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:31.086925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:31.087008 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:31.087161 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:31.087613 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:31.088757 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:31.090398 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:31.091083 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:31.091225 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:31.092540 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:31.092554 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:31.092560 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:31.092596 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:31.092655 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:31.092684 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:31.092794 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:31.092964 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 01:19:31.093072 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:19:31.096266 25822 raft_consensus.cc:504] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b70c10a3676643ddb5a9c471ec6833fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37223 } }
I20260806 01:19:31.096807 25822 leader_election.cc:302] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b70c10a3676643ddb5a9c471ec6833fb; no voters:
I20260806 01:19:31.097431 25822 leader_election.cc:288] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:31.097627 25835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:31.098459 25835 raft_consensus.cc:686] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [term 1 LEADER]: Becoming Leader. State: Replica: b70c10a3676643ddb5a9c471ec6833fb, State: Running, Role: LEADER
I20260806 01:19:31.099260 25835 consensus_queue.cc:237] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b70c10a3676643ddb5a9c471ec6833fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37223 } }
I20260806 01:19:31.099836 25822 sys_catalog.cc:564] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:31.103416 25839 sys_catalog.cc:455] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b70c10a3676643ddb5a9c471ec6833fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b70c10a3676643ddb5a9c471ec6833fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37223 } } }
I20260806 01:19:31.103411 25840 sys_catalog.cc:455] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b70c10a3676643ddb5a9c471ec6833fb. Latest consensus state: current_term: 1 leader_uuid: "b70c10a3676643ddb5a9c471ec6833fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b70c10a3676643ddb5a9c471ec6833fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37223 } } }
I20260806 01:19:31.103937 25840 sys_catalog.cc:458] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:31.103914 25839 sys_catalog.cc:458] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:31.105599 25854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:31.111021 25854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:31.120759 25854 catalog_manager.cc:1347] Generated new cluster ID: eec69059302c4f36be4c3077ae833d9f
I20260806 01:19:31.120891 25854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:31.140949 25854 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:31.142688 25854 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:31.152781 25854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b70c10a3676643ddb5a9c471ec6833fb: Generated new TSK 0
I20260806 01:19:31.153681 25854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:31.173260 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 25724
W20260806 01:19:31.174487 25817 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57371
W20260806 01:19:31.175175 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 25724: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:19:31.175329 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 25724
2026-08-06T01:19:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2406 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:31 dist-test-slave-7nm7 krb5kdc[25872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:31 dist-test-slave-7nm7 krb5kdc[25872](info): set up 2 sockets
Aug 06 01:19:31 dist-test-slave-7nm7 krb5kdc[25872](info): commencing operation
krb5kdc: starting...
W20260806 01:19:32.594336 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.378s user 0.003s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:32 dist-test-slave-7nm7 krb5kdc[25872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:19:32.947113 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43275
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:33.318190 25892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:33.318626 25892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:33.318799 25892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:33.326742 25892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:33.326911 25892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:33.327010 25892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:33.327117 25892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:33.327203 25892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:33.339310 25892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:33.340734 25892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:33.342129 25892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:33.354964 25900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:33.355252 25901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:33.355482 25905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:33.356724 25892 server_base.cc:1034] running on GCE node
I20260806 01:19:33.357826 25892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:33.359784 25892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:33.361018 25892 hybrid_clock.cc:648] HybridClock initialized: now 1785979173360952 us; error 62 us; skew 500 ppm
I20260806 01:19:33.361413 25892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:33 dist-test-slave-7nm7 krb5kdc[25872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979173, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:33.364661 25892 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:19:33.366390 25892 webserver.cc:460] Webserver started at http://127.0.0.1:42797/ using document root <none> and password file <none>
I20260806 01:19:33.367050 25892 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:33.367172 25892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:33.367443 25892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:33.370539 25892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "5e478718a51645c9818eef82a39bc77f"
format_stamp: "Formatted at 2026-08-06 01:19:33 on dist-test-slave-7nm7"
server_key: "bdd708e9baf6e438661e6432ef4b79bb"
server_key_iv: "2a3ea994be701e06c659f7723976541e"
server_key_version: "encryptionkey@0"
I20260806 01:19:33.371315 25892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "5e478718a51645c9818eef82a39bc77f"
format_stamp: "Formatted at 2026-08-06 01:19:33 on dist-test-slave-7nm7"
server_key: "bdd708e9baf6e438661e6432ef4b79bb"
server_key_iv: "2a3ea994be701e06c659f7723976541e"
server_key_version: "encryptionkey@0"
I20260806 01:19:33.376780 25892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:19:33.381400 25913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:33.382818 25892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:33.383014 25892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "5e478718a51645c9818eef82a39bc77f"
format_stamp: "Formatted at 2026-08-06 01:19:33 on dist-test-slave-7nm7"
server_key: "bdd708e9baf6e438661e6432ef4b79bb"
server_key_iv: "2a3ea994be701e06c659f7723976541e"
server_key_version: "encryptionkey@0"
I20260806 01:19:33.383247 25892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:33.389075 25892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:33.393204 25892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:33.393606 25892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:33.413132 25892 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43275
I20260806 01:19:33.413190 25993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43275 every 8 connection(s)
I20260806 01:19:33.414651 25892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:33.419762 25995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:33.421216 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 25892
I20260806 01:19:33.421409 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:33.422001 26546 external_mini_cluster.cc:1428] Setting key 97fd22c390dcce124c344e18c5615391
I20260806 01:19:33.432410 26001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:33.424129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35327 (local address 127.0.0.1:43275)
0806 01:19:33.425141 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:33.425156 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:33.425562 (+ 406us) server_negotiation.cc:407] Connection header received
0806 01:19:33.428182 (+ 2620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:33.428211 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:33.428680 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:33.429153 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:33.430204 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:33.430214 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:33.430260 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:33.430289 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:33.430361 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:33.430389 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:33.430644 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:33.430844 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 01:19:33.431058 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":480,"threads_started":1}
I20260806 01:19:33.434190 25995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f: Bootstrap starting.
I20260806 01:19:33.437410 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:33.437924 25995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:33.439148 25995 log.cc:824] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:33.443158 25995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f: No bootstrap required, opened a new log
I20260806 01:19:33.445461 26001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:33.442771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54409 (local address 127.0.0.1:43275)
0806 01:19:33.442965 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:33.442974 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:33.443355 (+ 381us) server_negotiation.cc:407] Connection header received
0806 01:19:33.443538 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:33.443542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:33.443594 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:33.443721 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:33.444687 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:33.444692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:33.444695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:33.444711 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:33.444759 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:33.444778 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:33.444919 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:33.445037 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 01:19:33.445201 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 01:19:33.451843 25995 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e478718a51645c9818eef82a39bc77f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43275 } }
I20260806 01:19:33.452239 25995 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:33.452349 25995 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e478718a51645c9818eef82a39bc77f, State: Initialized, Role: FOLLOWER
I20260806 01:19:33.452911 25995 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e478718a51645c9818eef82a39bc77f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43275 } }
I20260806 01:19:33.453155 25995 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:33.453305 25995 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:33.453487 25995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:33.455783 25995 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e478718a51645c9818eef82a39bc77f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43275 } }
I20260806 01:19:33.456269 25995 leader_election.cc:302] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e478718a51645c9818eef82a39bc77f; no voters:
I20260806 01:19:33.456893 25995 leader_election.cc:288] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:33.457049 26012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:33.458410 26012 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [term 1 LEADER]: Becoming Leader. State: Replica: 5e478718a51645c9818eef82a39bc77f, State: Running, Role: LEADER
I20260806 01:19:33.458992 26012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e478718a51645c9818eef82a39bc77f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43275 } }
I20260806 01:19:33.459157 25995 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:33.461838 26014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e478718a51645c9818eef82a39bc77f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e478718a51645c9818eef82a39bc77f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43275 } } }
I20260806 01:19:33.462317 26014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:33.461815 26015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e478718a51645c9818eef82a39bc77f. Latest consensus state: current_term: 1 leader_uuid: "5e478718a51645c9818eef82a39bc77f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e478718a51645c9818eef82a39bc77f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43275 } } }
I20260806 01:19:33.462446 26015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:33.463198 26023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:33.468691 26023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:33.477372 26023 catalog_manager.cc:1347] Generated new cluster ID: b82cdc115b374ea4aff03ce3e7c3bc89
I20260806 01:19:33.477509 26023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:33.490538 26023 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:33.490960 26023 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:33.500833 26023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e478718a51645c9818eef82a39bc77f: Generated new TSK 0
I20260806 01:19:33.501727 26023 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:33.514978 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 25892
W20260806 01:19:33.516175 25986 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35327
W20260806 01:19:33.516966 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 25892: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:19:33.517146 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 25892
2026-08-06T01:19:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:33 dist-test-slave-7nm7 krb5kdc[26044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:33 dist-test-slave-7nm7 krb5kdc[26044](info): set up 2 sockets
Aug 06 01:19:33 dist-test-slave-7nm7 krb5kdc[26044](info): commencing operation
krb5kdc: starting...
W20260806 01:19:34.922479 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.353s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:35 dist-test-slave-7nm7 krb5kdc[26044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:19:35.286072 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34987
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:35.671334 26066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:35.671643 26066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:35.671815 26066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:35.680310 26066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:35.680437 26066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:35.680543 26066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:35.680641 26066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:35.680721 26066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:35.691555 26066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:35.692914 26066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:35.694299 26066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:35.706337 26075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:35.706416 26076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:35.706385 26078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:35.707332 26066 server_base.cc:1034] running on GCE node
I20260806 01:19:35.708118 26066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:35.709625 26066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:35.710860 26066 hybrid_clock.cc:648] HybridClock initialized: now 1785979175710766 us; error 78 us; skew 500 ppm
I20260806 01:19:35.711216 26066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:35 dist-test-slave-7nm7 krb5kdc[26044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:35.714299 26066 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:19:35.715700 26066 webserver.cc:460] Webserver started at http://127.0.0.1:44069/ using document root <none> and password file <none>
I20260806 01:19:35.716347 26066 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:35.716496 26066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:35.716791 26066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:35.720011 26066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "43afc9d390f242aca269afe69a180549"
format_stamp: "Formatted at 2026-08-06 01:19:35 on dist-test-slave-7nm7"
server_key: "389521e88db0c6526891c5b603348888"
server_key_iv: "02154a16e468277793fba988d70a57ec"
server_key_version: "encryptionkey@0"
I20260806 01:19:35.720798 26066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "43afc9d390f242aca269afe69a180549"
format_stamp: "Formatted at 2026-08-06 01:19:35 on dist-test-slave-7nm7"
server_key: "389521e88db0c6526891c5b603348888"
server_key_iv: "02154a16e468277793fba988d70a57ec"
server_key_version: "encryptionkey@0"
I20260806 01:19:35.725888 26066 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:19:35.729275 26087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:35.730607 26066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:19:35.730782 26066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "43afc9d390f242aca269afe69a180549"
format_stamp: "Formatted at 2026-08-06 01:19:35 on dist-test-slave-7nm7"
server_key: "389521e88db0c6526891c5b603348888"
server_key_iv: "02154a16e468277793fba988d70a57ec"
server_key_version: "encryptionkey@0"
I20260806 01:19:35.730971 26066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:35.744174 26066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:35.748075 26066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:35.748353 26066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:35.767279 26066 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34987
I20260806 01:19:35.767313 26167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34987 every 8 connection(s)
I20260806 01:19:35.768704 26066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:35.772444 26168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:35.774521 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 26066
I20260806 01:19:35.774703 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:35.775173 26546 external_mini_cluster.cc:1428] Setting key 12bf0bc2a79aec7842bbef9c291ea2a2
I20260806 01:19:35.785899 26168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549: Bootstrap starting.
I20260806 01:19:35.789631 26168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:35.790696 26168 log.cc:824] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549: Log is configured to *not* fsync() on all Append() calls
Aug 06 01:19:35 dist-test-slave-7nm7 krb5kdc[26044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 01:19:35.794219 26168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549: No bootstrap required, opened a new log
I20260806 01:19:35.802673 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:35.777120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:58711 (local address 127.0.0.1:34987)
0806 01:19:35.777918 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:35.777938 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:35.778002 (+ 64us) server_negotiation.cc:407] Connection header received
0806 01:19:35.780528 (+ 2526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:35.780555 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:35.781047 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:35.781472 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:35.783147 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:35.785207 (+ 2060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:35.786807 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:35.787888 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:35.788077 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:35.792748 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:35.792759 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:35.792778 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:35.792827 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:35.796863 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:35.797696 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:35.797704 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:35.797712 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:35.797932 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:35.798616 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:35.798626 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:35.798630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:35.799518 (+ 888us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:35.799860 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:35.800252 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 01:19:35.800692 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":294,"threads_started":1}
I20260806 01:19:35.806181 26168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43afc9d390f242aca269afe69a180549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34987 } }
I20260806 01:19:35.806638 26168 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:35.806764 26168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43afc9d390f242aca269afe69a180549, State: Initialized, Role: FOLLOWER
I20260806 01:19:35.806798 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:35.807346 26168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43afc9d390f242aca269afe69a180549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34987 } }
I20260806 01:19:35.807588 26168 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:35.807744 26168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:35.807915 26168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:35.810452 26168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43afc9d390f242aca269afe69a180549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34987 } }
I20260806 01:19:35.811002 26168 leader_election.cc:302] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43afc9d390f242aca269afe69a180549; no voters:
I20260806 01:19:35.811702 26168 leader_election.cc:288] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:35.811740 26181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:35.812405 26181 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [term 1 LEADER]: Becoming Leader. State: Replica: 43afc9d390f242aca269afe69a180549, State: Running, Role: LEADER
I20260806 01:19:35.813136 26181 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43afc9d390f242aca269afe69a180549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34987 } }
I20260806 01:19:35.815922 26168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:35.815816 26183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43afc9d390f242aca269afe69a180549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43afc9d390f242aca269afe69a180549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34987 } } }
I20260806 01:19:35.816253 26183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:35.816769 26184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43afc9d390f242aca269afe69a180549. Latest consensus state: current_term: 1 leader_uuid: "43afc9d390f242aca269afe69a180549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43afc9d390f242aca269afe69a180549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34987 } } }
I20260806 01:19:35.817044 26184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:35.822037 26188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:35.828604 26188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:35.833156 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:35.811264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:50887 (local address 127.0.0.1:34987)
0806 01:19:35.811411 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:35.811430 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:35.813501 (+ 2071us) server_negotiation.cc:407] Connection header received
0806 01:19:35.813701 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:35.813707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:35.813778 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:35.813877 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:35.815907 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:35.817266 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:35.819741 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:35.820261 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:35.820330 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:35.823546 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:35.823559 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:35.823574 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:35.823616 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:35.827983 (+ 4367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:35.829067 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:35.829073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:35.829077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:35.829241 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:35.829884 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:35.829889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:35.829892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:35.830454 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:35.832208 (+ 1754us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:35.832582 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 01:19:35.832816 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 01:19:35.839314 26188 catalog_manager.cc:1347] Generated new cluster ID: eb98685afac1409984d1c3d7cb8dd009
I20260806 01:19:35.839447 26188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:35.865459 26188 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:35.867414 26188 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:35.879024 26188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43afc9d390f242aca269afe69a180549: Generated new TSK 0
I20260806 01:19:35.880143 26188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:35.895367 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 26066
I20260806 01:19:35.897503 26158 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:35.995708 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 26066
W20260806 01:19:36.021938 26042 connection.cc:535] client connection to 127.0.0.1:34987 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34987): Connection reset by peer (error 104)
2026-08-06T01:19:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:36 dist-test-slave-7nm7 krb5kdc[26214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:36 dist-test-slave-7nm7 krb5kdc[26214](info): set up 2 sockets
Aug 06 01:19:36 dist-test-slave-7nm7 krb5kdc[26214](info): commencing operation
krb5kdc: starting...
W20260806 01:19:37.388721 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.347s user 0.000s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:37 dist-test-slave-7nm7 krb5kdc[26214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:19:37.758877 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33995
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:38.130606 26231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:38.131026 26231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:38.131198 26231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:38.139362 26231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:38.139628 26231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:38.139739 26231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:38.139858 26231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:38.139943 26231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:38.150733 26231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:38.152159 26231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:38.153514 26231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:38.165693 26237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:38.165663 26238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:38.165624 26240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:38.166427 26231 server_base.cc:1034] running on GCE node
I20260806 01:19:38.167232 26231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:38.168919 26231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:38.170157 26231 hybrid_clock.cc:648] HybridClock initialized: now 1785979178170114 us; error 31 us; skew 500 ppm
I20260806 01:19:38.170537 26231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:38 dist-test-slave-7nm7 krb5kdc[26214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979178, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:38.173719 26231 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:19:38.175316 26231 webserver.cc:460] Webserver started at http://127.0.0.1:43797/ using document root <none> and password file <none>
I20260806 01:19:38.176002 26231 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:38.176157 26231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:38.176465 26231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:38.179575 26231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "f595b88309cb4da48ed28f3cd194480a"
format_stamp: "Formatted at 2026-08-06 01:19:38 on dist-test-slave-7nm7"
server_key: "1f8a6a4fc157a42d41a48a18bc1c6b1c"
server_key_iv: "dcc84a46a3781e561b7d4569d9c7edff"
server_key_version: "encryptionkey@0"
I20260806 01:19:38.180365 26231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "f595b88309cb4da48ed28f3cd194480a"
format_stamp: "Formatted at 2026-08-06 01:19:38 on dist-test-slave-7nm7"
server_key: "1f8a6a4fc157a42d41a48a18bc1c6b1c"
server_key_iv: "dcc84a46a3781e561b7d4569d9c7edff"
server_key_version: "encryptionkey@0"
I20260806 01:19:38.185781 26231 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 01:19:38.189445 26255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:38.190833 26231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:38.191037 26231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "f595b88309cb4da48ed28f3cd194480a"
format_stamp: "Formatted at 2026-08-06 01:19:38 on dist-test-slave-7nm7"
server_key: "1f8a6a4fc157a42d41a48a18bc1c6b1c"
server_key_iv: "dcc84a46a3781e561b7d4569d9c7edff"
server_key_version: "encryptionkey@0"
I20260806 01:19:38.191253 26231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:38.215646 26231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:38.220422 26231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:38.220762 26231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:38.242882 26231 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33995
I20260806 01:19:38.242913 26329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33995 every 8 connection(s)
I20260806 01:19:38.244518 26231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:38.245690 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 26231
I20260806 01:19:38.245904 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:38.246485 26546 external_mini_cluster.cc:1428] Setting key 35a04065eb7d8e076b8ea03296364136
I20260806 01:19:38.250197 26331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:38.262151 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:38.248439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:44347 (local address 127.0.0.1:33995)
0806 01:19:38.249297 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:38.249309 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:38.249351 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:19:38.251352 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:38.251377 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:38.251779 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:38.252115 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:38.252817 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:38.254708 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:38.256867 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:38.257839 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:38.257995 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:38.259477 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:38.259485 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:38.259501 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:38.260251 (+ 750us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:38.260340 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:38.260368 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:38.260509 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:38.260846 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 01:19:38.261072 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":380,"threads_started":1}
I20260806 01:19:38.263669 26331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a: Bootstrap starting.
I20260806 01:19:38.266265 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:38.267148 26331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:38.268419 26331 log.cc:824] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:38.271770 26331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a: No bootstrap required, opened a new log
I20260806 01:19:38.281056 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:38.272470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:39753 (local address 127.0.0.1:33995)
0806 01:19:38.272705 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:38.272713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:38.273428 (+ 715us) server_negotiation.cc:407] Connection header received
0806 01:19:38.273597 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:38.273603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:38.273693 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:38.273830 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:38.274672 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:38.276340 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:38.277770 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:38.278522 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:38.278653 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:38.279917 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:38.279924 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:38.279928 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:38.279954 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:38.280001 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:38.280022 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:38.280142 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:38.280370 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 01:19:38.280438 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:19:38.283592 26331 raft_consensus.cc:348] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f595b88309cb4da48ed28f3cd194480a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33995 } }
I20260806 01:19:38.284018 26331 raft_consensus.cc:374] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:38.284193 26331 raft_consensus.cc:729] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f595b88309cb4da48ed28f3cd194480a, State: Initialized, Role: FOLLOWER
I20260806 01:19:38.284889 26331 consensus_queue.cc:260] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f595b88309cb4da48ed28f3cd194480a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33995 } }
I20260806 01:19:38.285125 26331 raft_consensus.cc:388] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:38.285262 26331 raft_consensus.cc:482] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:38.285450 26331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:38.287979 26331 raft_consensus.cc:504] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f595b88309cb4da48ed28f3cd194480a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33995 } }
I20260806 01:19:38.288529 26331 leader_election.cc:302] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f595b88309cb4da48ed28f3cd194480a; no voters:
I20260806 01:19:38.289237 26331 leader_election.cc:288] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:38.289513 26342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:38.290661 26342 raft_consensus.cc:686] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [term 1 LEADER]: Becoming Leader. State: Replica: f595b88309cb4da48ed28f3cd194480a, State: Running, Role: LEADER
I20260806 01:19:38.291523 26331 sys_catalog.cc:564] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:38.291349 26342 consensus_queue.cc:237] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f595b88309cb4da48ed28f3cd194480a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33995 } }
I20260806 01:19:38.295253 26344 sys_catalog.cc:455] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f595b88309cb4da48ed28f3cd194480a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f595b88309cb4da48ed28f3cd194480a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33995 } } }
I20260806 01:19:38.295677 26344 sys_catalog.cc:458] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:38.295619 26346 sys_catalog.cc:455] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f595b88309cb4da48ed28f3cd194480a. Latest consensus state: current_term: 1 leader_uuid: "f595b88309cb4da48ed28f3cd194480a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f595b88309cb4da48ed28f3cd194480a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33995 } } }
I20260806 01:19:38.296044 26346 sys_catalog.cc:458] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:38.296797 26358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:38.305164 26358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:38.318497 26358 catalog_manager.cc:1347] Generated new cluster ID: eaada70ac7674cffa5d7e0cb5517b2f9
I20260806 01:19:38.318629 26358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:38.333633 26358 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:38.335407 26358 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:38.345310 26358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f595b88309cb4da48ed28f3cd194480a: Generated new TSK 0
I20260806 01:19:38.346221 26358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:38.358767 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 26231
W20260806 01:19:38.360249 26325 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.43:44347
W20260806 01:19:38.361603 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 26231: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:19:38.361796 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 26231
W20260806 01:19:38.391746 26212 connection.cc:535] client connection to 127.0.0.1:33995 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33995): Connection reset by peer (error 104)
2026-08-06T01:19:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2369 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:38 dist-test-slave-7nm7 krb5kdc[26375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:38 dist-test-slave-7nm7 krb5kdc[26375](info): set up 2 sockets
Aug 06 01:19:38 dist-test-slave-7nm7 krb5kdc[26375](info): commencing operation
krb5kdc: starting...
W20260806 01:19:39.756949 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.346s user 0.000s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:39 dist-test-slave-7nm7 krb5kdc[26375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:19:40.113787 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:40.473577 26395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:40.473882 26395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:40.473981 26395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:40.482059 26395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:40.482209 26395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:40.482345 26395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:40.482419 26395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:40.482471 26395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:40.492965 26395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:40.494563 26395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:40.496475 26395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:40.510528 26409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:40.510605 26404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:40.510612 26405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:40.511127 26395 server_base.cc:1034] running on GCE node
I20260806 01:19:40.512044 26395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:40.513377 26395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:40.514590 26395 hybrid_clock.cc:648] HybridClock initialized: now 1785979180514519 us; error 54 us; skew 500 ppm
I20260806 01:19:40.514955 26395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:40 dist-test-slave-7nm7 krb5kdc[26375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979180, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:40.522802 26395 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:19:40.524497 26395 webserver.cc:460] Webserver started at http://127.0.0.1:37921/ using document root <none> and password file <none>
I20260806 01:19:40.525177 26395 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:40.525293 26395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:40.525547 26395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:40.528626 26395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "a9c4889e574e4bdf857a4ca5b2779ad9"
format_stamp: "Formatted at 2026-08-06 01:19:40 on dist-test-slave-7nm7"
server_key: "e8029d90ca8f78d12cb040f19206bdc9"
server_key_iv: "174a76fb47dbb97ecee20231fbd3dde5"
server_key_version: "encryptionkey@0"
I20260806 01:19:40.529361 26395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "a9c4889e574e4bdf857a4ca5b2779ad9"
format_stamp: "Formatted at 2026-08-06 01:19:40 on dist-test-slave-7nm7"
server_key: "e8029d90ca8f78d12cb040f19206bdc9"
server_key_iv: "174a76fb47dbb97ecee20231fbd3dde5"
server_key_version: "encryptionkey@0"
I20260806 01:19:40.535353 26395 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260806 01:19:40.539206 26423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:40.540612 26395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:19:40.540844 26395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "a9c4889e574e4bdf857a4ca5b2779ad9"
format_stamp: "Formatted at 2026-08-06 01:19:40 on dist-test-slave-7nm7"
server_key: "e8029d90ca8f78d12cb040f19206bdc9"
server_key_iv: "174a76fb47dbb97ecee20231fbd3dde5"
server_key_version: "encryptionkey@0"
I20260806 01:19:40.541074 26395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:40.547787 26395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:40.552613 26395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:40.552930 26395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:40.574662 26395 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44709
I20260806 01:19:40.574687 26497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44709 every 8 connection(s)
I20260806 01:19:40.576218 26395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:40.579234 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 26395
I20260806 01:19:40.579430 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:40.580003 26546 external_mini_cluster.cc:1428] Setting key c228b7bae0a552fb069a6adbb82c97e3
I20260806 01:19:40.580670 26500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:40.589362 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:40.582190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:48425 (local address 127.0.0.1:44709)
0806 01:19:40.583178 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:40.583192 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:40.583240 (+ 48us) server_negotiation.cc:407] Connection header received
0806 01:19:40.585776 (+ 2536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:40.585799 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:40.586158 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:40.586643 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:40.587763 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:40.587769 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:40.587781 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:40.587800 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:40.587864 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:40.587886 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:40.588151 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:40.588301 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 01:19:40.588471 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":443,"threads_started":1}
I20260806 01:19:40.593247 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:40.594834 26500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9: Bootstrap starting.
I20260806 01:19:40.599129 26500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:40.600370 26500 log.cc:824] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:40.600594 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:40.598099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:36607 (local address 127.0.0.1:44709)
0806 01:19:40.598322 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:40.598329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:40.598629 (+ 300us) server_negotiation.cc:407] Connection header received
0806 01:19:40.598764 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:40.598770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:40.598836 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:40.598969 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:40.599907 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:40.599916 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:40.599922 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:40.599940 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:40.599987 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:40.600017 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:40.600152 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:40.600217 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 01:19:40.600276 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:19:40.603261 26500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9: No bootstrap required, opened a new log
I20260806 01:19:40.610749 26500 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c4889e574e4bdf857a4ca5b2779ad9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44709 } }
I20260806 01:19:40.611135 26500 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:40.611250 26500 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c4889e574e4bdf857a4ca5b2779ad9, State: Initialized, Role: FOLLOWER
I20260806 01:19:40.611835 26500 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c4889e574e4bdf857a4ca5b2779ad9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44709 } }
I20260806 01:19:40.612067 26500 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:40.612224 26500 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:40.612397 26500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:40.614734 26500 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c4889e574e4bdf857a4ca5b2779ad9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44709 } }
I20260806 01:19:40.615268 26500 leader_election.cc:302] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9c4889e574e4bdf857a4ca5b2779ad9; no voters:
I20260806 01:19:40.615851 26500 leader_election.cc:288] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:40.616015 26513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:40.617590 26513 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [term 1 LEADER]: Becoming Leader. State: Replica: a9c4889e574e4bdf857a4ca5b2779ad9, State: Running, Role: LEADER
I20260806 01:19:40.618103 26513 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c4889e574e4bdf857a4ca5b2779ad9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44709 } }
I20260806 01:19:40.618377 26500 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:40.621001 26515 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9c4889e574e4bdf857a4ca5b2779ad9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c4889e574e4bdf857a4ca5b2779ad9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44709 } } }
I20260806 01:19:40.621266 26516 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9c4889e574e4bdf857a4ca5b2779ad9. Latest consensus state: current_term: 1 leader_uuid: "a9c4889e574e4bdf857a4ca5b2779ad9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c4889e574e4bdf857a4ca5b2779ad9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44709 } } }
I20260806 01:19:40.621455 26515 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:40.621526 26516 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:40.627526 26523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:40.633653 26523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:40.651150 26523 catalog_manager.cc:1347] Generated new cluster ID: e994450aad6d419593f5929e01f66d14
I20260806 01:19:40.651288 26523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:40.687745 26523 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:40.688208 26523 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:40.696380 26523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9c4889e574e4bdf857a4ca5b2779ad9: Generated new TSK 0
I20260806 01:19:40.697256 26523 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:40.711752 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 26395
W20260806 01:19:40.712903 26491 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.43:48425
W20260806 01:19:40.713650 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 26395: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:19:40.713795 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 26395
2026-08-06T01:19:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:40 dist-test-slave-7nm7 krb5kdc[26545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:40 dist-test-slave-7nm7 krb5kdc[26545](info): set up 2 sockets
Aug 06 01:19:40 dist-test-slave-7nm7 krb5kdc[26545](info): commencing operation
krb5kdc: starting...
W20260806 01:19:42.087852 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.002s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:42 dist-test-slave-7nm7 krb5kdc[26545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:19:42.449968 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:42.822918 26572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:42.823212 26572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:42.823307 26572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:42.830585 26572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:42.830706 26572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:42.830753 26572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:42.830821 26572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:42.830860 26572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:42.841485 26572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43977
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:42.842919 26572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:42.844257 26572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:42.856647 26580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:42.856876 26581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:42.856600 26585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:42.857883 26572 server_base.cc:1034] running on GCE node
I20260806 01:19:42.858774 26572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:42.860263 26572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:42.861526 26572 hybrid_clock.cc:648] HybridClock initialized: now 1785979182861469 us; error 47 us; skew 500 ppm
I20260806 01:19:42.861887 26572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:42 dist-test-slave-7nm7 krb5kdc[26545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979182, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:42.865098 26572 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:19:42.866621 26572 webserver.cc:460] Webserver started at http://127.0.0.1:35731/ using document root <none> and password file <none>
I20260806 01:19:42.867311 26572 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:42.867446 26572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:42.867774 26572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:42.870869 26572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "e06bb473c67c41f08bed07c54b45c52e"
format_stamp: "Formatted at 2026-08-06 01:19:42 on dist-test-slave-7nm7"
server_key: "651d49ee7746e555ffc8196905c0500c"
server_key_iv: "4a9b35f9e824f409f9c5fedbc8023808"
server_key_version: "encryptionkey@0"
I20260806 01:19:42.871663 26572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "e06bb473c67c41f08bed07c54b45c52e"
format_stamp: "Formatted at 2026-08-06 01:19:42 on dist-test-slave-7nm7"
server_key: "651d49ee7746e555ffc8196905c0500c"
server_key_iv: "4a9b35f9e824f409f9c5fedbc8023808"
server_key_version: "encryptionkey@0"
I20260806 01:19:42.876995 26572 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:19:42.881047 26595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:42.882546 26572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:42.882746 26572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "e06bb473c67c41f08bed07c54b45c52e"
format_stamp: "Formatted at 2026-08-06 01:19:42 on dist-test-slave-7nm7"
server_key: "651d49ee7746e555ffc8196905c0500c"
server_key_iv: "4a9b35f9e824f409f9c5fedbc8023808"
server_key_version: "encryptionkey@0"
I20260806 01:19:42.882954 26572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:42.906737 26572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:42.911218 26572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:42.911540 26572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:42.932821 26572 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43977
I20260806 01:19:42.932842 26671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43977 every 8 connection(s)
I20260806 01:19:42.934381 26572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:42.938504 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 26572
I20260806 01:19:42.938637 26672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:42.938717 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:42.939294 26546 external_mini_cluster.cc:1428] Setting key 4f3763c45d6ccf7fd5e233432fea7a26
I20260806 01:19:42.953601 26672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e: Bootstrap starting.
Aug 06 01:19:42 dist-test-slave-7nm7 krb5kdc[26545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 01:19:42.957000 26672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:42.958449 26672 log.cc:824] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:42.961598 26672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e: No bootstrap required, opened a new log
I20260806 01:19:42.966197 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:42.941222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:34201 (local address 127.0.0.1:43977)
0806 01:19:42.942065 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:42.942078 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:42.942447 (+ 369us) server_negotiation.cc:407] Connection header received
0806 01:19:42.944900 (+ 2453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:42.944930 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:42.945507 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:42.945981 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:42.947772 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:42.949072 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:42.950383 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:42.951084 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:42.951324 (+ 240us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:42.955833 (+ 4509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:42.955841 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:42.955854 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:42.955891 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:42.960638 (+ 4747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:42.961742 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:42.961749 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:42.961755 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:42.961944 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:42.963037 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:42.963041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:42.963043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:42.963686 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:42.964106 (+ 420us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:42.964770 (+ 664us) server_negotiation.cc:299] Negotiation successful
0806 01:19:42.965106 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":383,"threads_started":1}
I20260806 01:19:42.970660 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:42.970497 26672 raft_consensus.cc:348] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06bb473c67c41f08bed07c54b45c52e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43977 } }
I20260806 01:19:42.970945 26672 raft_consensus.cc:374] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:42.971057 26672 raft_consensus.cc:729] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06bb473c67c41f08bed07c54b45c52e, State: Initialized, Role: FOLLOWER
I20260806 01:19:42.972266 26672 consensus_queue.cc:260] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06bb473c67c41f08bed07c54b45c52e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43977 } }
I20260806 01:19:42.972644 26672 raft_consensus.cc:388] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:42.972910 26672 raft_consensus.cc:482] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:42.973215 26672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:42.976370 26672 raft_consensus.cc:504] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06bb473c67c41f08bed07c54b45c52e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43977 } }
I20260806 01:19:42.976989 26672 leader_election.cc:302] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e06bb473c67c41f08bed07c54b45c52e; no voters:
I20260806 01:19:42.977598 26672 leader_election.cc:288] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:42.977773 26686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:42.979352 26686 raft_consensus.cc:686] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [term 1 LEADER]: Becoming Leader. State: Replica: e06bb473c67c41f08bed07c54b45c52e, State: Running, Role: LEADER
I20260806 01:19:42.980037 26686 consensus_queue.cc:237] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06bb473c67c41f08bed07c54b45c52e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43977 } }
I20260806 01:19:42.980289 26672 sys_catalog.cc:564] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:42.983798 26689 sys_catalog.cc:455] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e06bb473c67c41f08bed07c54b45c52e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06bb473c67c41f08bed07c54b45c52e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43977 } } }
I20260806 01:19:42.984366 26689 sys_catalog.cc:458] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:42.985783 26690 sys_catalog.cc:455] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e06bb473c67c41f08bed07c54b45c52e. Latest consensus state: current_term: 1 leader_uuid: "e06bb473c67c41f08bed07c54b45c52e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06bb473c67c41f08bed07c54b45c52e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43977 } } }
I20260806 01:19:42.986284 26690 sys_catalog.cc:458] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:42.990510 26694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:42.996892 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:42.975496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:51227 (local address 127.0.0.1:43977)
0806 01:19:42.975682 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:42.975688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:42.976225 (+ 537us) server_negotiation.cc:407] Connection header received
0806 01:19:42.976369 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:42.976374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:42.976460 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:42.976577 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:42.978900 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:42.980233 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:42.982298 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:42.983261 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:42.983425 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:42.988300 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:42.988307 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:42.988312 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:42.988361 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:42.992408 (+ 4047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:42.993316 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:42.993323 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:42.993329 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:42.993488 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:42.994306 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:42.994312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:42.994315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:42.994788 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:42.996141 (+ 1353us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:42.996355 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 01:19:42.996544 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:19:42.998701 26694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:43.010973 26694 catalog_manager.cc:1347] Generated new cluster ID: 8b57d0fafda644f390c2fdccba1e4d9f
I20260806 01:19:43.011116 26694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:43.034621 26694 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:43.035969 26694 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:43.047572 26694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e06bb473c67c41f08bed07c54b45c52e: Generated new TSK 0
I20260806 01:19:43.048512 26694 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:43.054900 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 26572
I20260806 01:19:43.056340 26667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:43.151587 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 26572
W20260806 01:19:43.177510 26543 connection.cc:535] client connection to 127.0.0.1:43977 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43977): Connection reset by peer (error 104)
2026-08-06T01:19:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:43 dist-test-slave-7nm7 krb5kdc[26718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:43 dist-test-slave-7nm7 krb5kdc[26718](info): set up 2 sockets
Aug 06 01:19:43 dist-test-slave-7nm7 krb5kdc[26718](info): commencing operation
krb5kdc: starting...
W20260806 01:19:44.569656 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.369s user 0.003s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:44 dist-test-slave-7nm7 krb5kdc[26718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:19:44.950479 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:45.332986 26745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:45.333335 26745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:45.333515 26745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:45.340826 26745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:45.341001 26745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:45.341110 26745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:45.341223 26745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:45.341303 26745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:45.352455 26745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44757
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:45.353813 26745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:45.355170 26745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:45.367914 26756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:45.368005 26754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:45.367981 26758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:45.369177 26745 server_base.cc:1034] running on GCE node
I20260806 01:19:45.370039 26745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:45.371592 26745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:45.372867 26745 hybrid_clock.cc:648] HybridClock initialized: now 1785979185372822 us; error 58 us; skew 500 ppm
I20260806 01:19:45.373281 26745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:45 dist-test-slave-7nm7 krb5kdc[26718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979185, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:45.376794 26745 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:19:45.378715 26745 webserver.cc:460] Webserver started at http://127.0.0.1:45719/ using document root <none> and password file <none>
I20260806 01:19:45.379451 26745 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:45.379621 26745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:45.379957 26745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:45.383302 26745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "a6f5500c0e6c4c9c8aebfe9d1b4d4765"
format_stamp: "Formatted at 2026-08-06 01:19:45 on dist-test-slave-7nm7"
server_key: "1bb3df7ebeb4c69f70d6f6e32065fbf9"
server_key_iv: "d16f879f3e36b39df29df41b89a51683"
server_key_version: "encryptionkey@0"
I20260806 01:19:45.384133 26745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "a6f5500c0e6c4c9c8aebfe9d1b4d4765"
format_stamp: "Formatted at 2026-08-06 01:19:45 on dist-test-slave-7nm7"
server_key: "1bb3df7ebeb4c69f70d6f6e32065fbf9"
server_key_iv: "d16f879f3e36b39df29df41b89a51683"
server_key_version: "encryptionkey@0"
I20260806 01:19:45.389750 26745 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 01:19:45.393538 26767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:45.394862 26745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:45.395046 26745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "a6f5500c0e6c4c9c8aebfe9d1b4d4765"
format_stamp: "Formatted at 2026-08-06 01:19:45 on dist-test-slave-7nm7"
server_key: "1bb3df7ebeb4c69f70d6f6e32065fbf9"
server_key_iv: "d16f879f3e36b39df29df41b89a51683"
server_key_version: "encryptionkey@0"
I20260806 01:19:45.395252 26745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:45.408066 26745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:45.412623 26745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:45.412981 26745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:45.436059 26745 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44757
I20260806 01:19:45.436079 26837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44757 every 8 connection(s)
I20260806 01:19:45.437676 26745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:45.439419 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 26745
I20260806 01:19:45.439632 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:45.440114 26546 external_mini_cluster.cc:1428] Setting key 3199f554949eecb55afcdcc90a4fd1d3
I20260806 01:19:45.443238 26843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:45.458307 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:45.442309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:55163 (local address 127.0.0.1:44757)
0806 01:19:45.443362 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:45.443378 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:45.443430 (+ 52us) server_negotiation.cc:407] Connection header received
0806 01:19:45.446140 (+ 2710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:45.446173 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:45.446772 (+ 599us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:45.447259 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:45.447847 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:45.450541 (+ 2694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:45.452108 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:45.453180 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:45.453381 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:45.454746 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:45.454757 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:45.454778 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:45.455554 (+ 776us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:45.455671 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:45.455703 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:45.455874 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:45.456538 (+ 664us) server_negotiation.cc:299] Negotiation successful
0806 01:19:45.456738 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":510,"threads_started":1}
I20260806 01:19:45.459621 26843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765: Bootstrap starting.
I20260806 01:19:45.463063 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:45.463599 26843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:45.464895 26843 log.cc:824] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:45.469483 26843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765: No bootstrap required, opened a new log
I20260806 01:19:45.479606 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:45.471979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:43203 (local address 127.0.0.1:44757)
0806 01:19:45.472216 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:45.472226 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:45.472735 (+ 509us) server_negotiation.cc:407] Connection header received
0806 01:19:45.472959 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:45.472963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:45.473041 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:45.473245 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:45.473883 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:45.474798 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:45.476695 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:45.477220 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:45.477332 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:45.478694 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:45.478710 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:45.478713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:45.478731 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:45.478805 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:45.478821 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:45.478931 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:45.479155 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 01:19:45.479241 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:19:45.480465 26843 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f5500c0e6c4c9c8aebfe9d1b4d4765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } }
I20260806 01:19:45.480944 26843 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:45.481062 26843 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f5500c0e6c4c9c8aebfe9d1b4d4765, State: Initialized, Role: FOLLOWER
I20260806 01:19:45.482411 26843 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f5500c0e6c4c9c8aebfe9d1b4d4765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } }
I20260806 01:19:45.482702 26843 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:45.482858 26843 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:45.483037 26843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:45.485308 26843 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f5500c0e6c4c9c8aebfe9d1b4d4765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } }
I20260806 01:19:45.485823 26843 leader_election.cc:302] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6f5500c0e6c4c9c8aebfe9d1b4d4765; no voters:
I20260806 01:19:45.486572 26843 leader_election.cc:288] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:45.486701 26859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:45.487520 26859 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [term 1 LEADER]: Becoming Leader. State: Replica: a6f5500c0e6c4c9c8aebfe9d1b4d4765, State: Running, Role: LEADER
I20260806 01:19:45.488111 26859 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f5500c0e6c4c9c8aebfe9d1b4d4765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } }
I20260806 01:19:45.489029 26843 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:45.491103 26861 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6f5500c0e6c4c9c8aebfe9d1b4d4765. Latest consensus state: current_term: 1 leader_uuid: "a6f5500c0e6c4c9c8aebfe9d1b4d4765" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f5500c0e6c4c9c8aebfe9d1b4d4765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } } }
I20260806 01:19:45.491591 26861 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:45.494614 26860 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6f5500c0e6c4c9c8aebfe9d1b4d4765" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f5500c0e6c4c9c8aebfe9d1b4d4765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44757 } } }
I20260806 01:19:45.495103 26860 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:45.495280 26867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:45.501816 26867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:45.512543 26867 catalog_manager.cc:1347] Generated new cluster ID: 79be51ec7f1f4fa9a5e66d65685847ad
I20260806 01:19:45.512691 26867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:45.559793 26867 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:45.561992 26867 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:45.574280 26867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6f5500c0e6c4c9c8aebfe9d1b4d4765: Generated new TSK 0
I20260806 01:19:45.575353 26867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:45.600859 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 26745
W20260806 01:19:45.602741 26828 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.43:55163
W20260806 01:19:45.603681 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 26745: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:19:45.603886 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 26745
W20260806 01:19:45.633222 26715 connection.cc:535] client connection to 127.0.0.1:44757 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44757): Connection reset by peer (error 104)
2026-08-06T01:19:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2452 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:45 dist-test-slave-7nm7 krb5kdc[26890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:45 dist-test-slave-7nm7 krb5kdc[26890](info): set up 2 sockets
Aug 06 01:19:45 dist-test-slave-7nm7 krb5kdc[26890](info): commencing operation
krb5kdc: starting...
W20260806 01:19:47.007340 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.353s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:47 dist-test-slave-7nm7 krb5kdc[26890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:19:47.385210 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:47.781060 26904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:47.781581 26904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:47.781769 26904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:47.789825 26904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:47.790027 26904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:47.790158 26904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:47.790298 26904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:47.790398 26904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:47.800630 26904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39897
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:47.802459 26904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:47.803869 26904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:47.816633 26916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:47.816749 26911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:47.816749 26914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:47.817795 26904 server_base.cc:1034] running on GCE node
I20260806 01:19:47.818684 26904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:47.820137 26904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:47.821362 26904 hybrid_clock.cc:648] HybridClock initialized: now 1785979187821316 us; error 31 us; skew 500 ppm
I20260806 01:19:47.821734 26904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:47 dist-test-slave-7nm7 krb5kdc[26890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979187, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:47.825162 26904 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:19:47.826848 26904 webserver.cc:460] Webserver started at http://127.0.0.1:43405/ using document root <none> and password file <none>
I20260806 01:19:47.827543 26904 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:47.827692 26904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:47.828008 26904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:47.831113 26904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "e1e9f046832041d697d8342a63a62794"
format_stamp: "Formatted at 2026-08-06 01:19:47 on dist-test-slave-7nm7"
server_key: "d84f93500768e24fa4a5877d702b0091"
server_key_iv: "5262e0234d2dc990320b2a59943e8e39"
server_key_version: "encryptionkey@0"
I20260806 01:19:47.831895 26904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "e1e9f046832041d697d8342a63a62794"
format_stamp: "Formatted at 2026-08-06 01:19:47 on dist-test-slave-7nm7"
server_key: "d84f93500768e24fa4a5877d702b0091"
server_key_iv: "5262e0234d2dc990320b2a59943e8e39"
server_key_version: "encryptionkey@0"
I20260806 01:19:47.837615 26904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:19:47.841238 26923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:47.842876 26904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:47.843091 26904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "e1e9f046832041d697d8342a63a62794"
format_stamp: "Formatted at 2026-08-06 01:19:47 on dist-test-slave-7nm7"
server_key: "d84f93500768e24fa4a5877d702b0091"
server_key_iv: "5262e0234d2dc990320b2a59943e8e39"
server_key_version: "encryptionkey@0"
I20260806 01:19:47.843338 26904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:47.849347 26904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:47.853955 26904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:47.854312 26904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:47.874571 26904 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39897
I20260806 01:19:47.874624 27005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39897 every 8 connection(s)
I20260806 01:19:47.875985 26904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:47.879945 27006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:47.883744 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 26904
I20260806 01:19:47.883944 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:47.884511 26546 external_mini_cluster.cc:1428] Setting key f265b97a2d42c8658e8fad575a012abb
I20260806 01:19:47.892017 27009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:47.886394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:36661 (local address 127.0.0.1:39897)
0806 01:19:47.887185 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:47.887197 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:47.887237 (+ 40us) server_negotiation.cc:407] Connection header received
0806 01:19:47.888820 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:47.888840 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:47.889172 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:47.889475 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:47.890552 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:47.890558 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:47.890569 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:47.890593 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:47.890647 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:47.890665 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:47.890837 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:47.890934 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 01:19:47.891135 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":360,"threads_started":1}
I20260806 01:19:47.892809 27006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794: Bootstrap starting.
I20260806 01:19:47.895570 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:47.896366 27006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:47.897622 27006 log.cc:824] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:47.900550 27006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794: No bootstrap required, opened a new log
I20260806 01:19:47.903664 27009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:47.900872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.43:49999 (local address 127.0.0.1:39897)
0806 01:19:47.901095 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:47.901102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:47.901626 (+ 524us) server_negotiation.cc:407] Connection header received
0806 01:19:47.901792 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:47.901795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:47.901838 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:47.901920 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:47.902914 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:47.902917 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:47.902920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:47.902935 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:47.902975 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:47.902989 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:47.903081 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:47.903263 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 01:19:47.903360 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:19:47.909865 27006 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e9f046832041d697d8342a63a62794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260806 01:19:47.910296 27006 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:47.910424 27006 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1e9f046832041d697d8342a63a62794, State: Initialized, Role: FOLLOWER
I20260806 01:19:47.911060 27006 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e9f046832041d697d8342a63a62794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260806 01:19:47.911305 27006 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:47.911445 27006 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:47.911638 27006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:47.914409 27006 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e9f046832041d697d8342a63a62794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260806 01:19:47.914986 27006 leader_election.cc:302] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1e9f046832041d697d8342a63a62794; no voters:
I20260806 01:19:47.915616 27006 leader_election.cc:288] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:47.915781 27020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:47.916488 27020 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [term 1 LEADER]: Becoming Leader. State: Replica: e1e9f046832041d697d8342a63a62794, State: Running, Role: LEADER
I20260806 01:19:47.917016 27020 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e9f046832041d697d8342a63a62794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260806 01:19:47.917773 27006 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:47.920426 27022 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1e9f046832041d697d8342a63a62794. Latest consensus state: current_term: 1 leader_uuid: "e1e9f046832041d697d8342a63a62794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e9f046832041d697d8342a63a62794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } } }
I20260806 01:19:47.920957 27022 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:47.920095 27021 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1e9f046832041d697d8342a63a62794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e9f046832041d697d8342a63a62794" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } } }
I20260806 01:19:47.922030 27021 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:47.923532 27038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:47.929162 27038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:47.937537 27038 catalog_manager.cc:1347] Generated new cluster ID: 416c51e3f1ae4e1abc4c7344e6169b0a
I20260806 01:19:47.937656 27038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:47.948701 27038 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:47.949160 27038 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:47.956038 27038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1e9f046832041d697d8342a63a62794: Generated new TSK 0
I20260806 01:19:47.956826 27038 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:47.969863 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 26904
W20260806 01:19:47.971338 26996 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.43:36661
W20260806 01:19:47.972177 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 26904: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:19:47.972390 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 26904
2026-08-06T01:19:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2369 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:48 dist-test-slave-7nm7 krb5kdc[27055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:48 dist-test-slave-7nm7 krb5kdc[27055](info): set up 2 sockets
Aug 06 01:19:48 dist-test-slave-7nm7 krb5kdc[27055](info): commencing operation
krb5kdc: starting...
W20260806 01:19:49.402451 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.379s user 0.004s sys 0.065s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:49 dist-test-slave-7nm7 krb5kdc[27055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:19:49.777338 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:38901
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:35561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:38901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:50.161725 27077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:50.162010 27077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:50.162202 27077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:50.170006 27077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:50.170295 27077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:50.170421 27077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:50.170545 27077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:50.170629 27077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:50.181249 27077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:35561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:38901
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:38901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:50.182674 27077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:50.183986 27077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:50.196393 27083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:50.196488 27084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:50.196429 27086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:50.197438 27077 server_base.cc:1034] running on GCE node
I20260806 01:19:50.198393 27077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:50.199874 27077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:50.201146 27077 hybrid_clock.cc:648] HybridClock initialized: now 1785979190201088 us; error 55 us; skew 500 ppm
I20260806 01:19:50.201555 27077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:50 dist-test-slave-7nm7 krb5kdc[27055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979190, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:50.205016 27077 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:19:50.206702 27077 webserver.cc:460] Webserver started at http://127.25.236.190:44195/ using document root <none> and password file <none>
I20260806 01:19:50.207422 27077 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:50.207577 27077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:50.207887 27077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:50.211191 27077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "5887d0642d54434bab4864af4f5457f2"
format_stamp: "Formatted at 2026-08-06 01:19:50 on dist-test-slave-7nm7"
server_key: "751a64e63067e2dca6d8f7d5ab927f5c"
server_key_iv: "f5ea9408ac32411a1042bb779f61cea5"
server_key_version: "encryptionkey@0"
I20260806 01:19:50.212055 27077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "5887d0642d54434bab4864af4f5457f2"
format_stamp: "Formatted at 2026-08-06 01:19:50 on dist-test-slave-7nm7"
server_key: "751a64e63067e2dca6d8f7d5ab927f5c"
server_key_iv: "f5ea9408ac32411a1042bb779f61cea5"
server_key_version: "encryptionkey@0"
I20260806 01:19:50.217909 27077 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:19:50.221858 27098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:50.223336 27077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:19:50.223544 27077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "5887d0642d54434bab4864af4f5457f2"
format_stamp: "Formatted at 2026-08-06 01:19:50 on dist-test-slave-7nm7"
server_key: "751a64e63067e2dca6d8f7d5ab927f5c"
server_key_iv: "f5ea9408ac32411a1042bb779f61cea5"
server_key_version: "encryptionkey@0"
I20260806 01:19:50.223755 27077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:50.264247 27077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:50.269414 27077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:50.269739 27077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:50.292552 27077 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:38901
I20260806 01:19:50.292572 27171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:38901 every 8 connection(s)
I20260806 01:19:50.294055 27077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:50.296844 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 27077
I20260806 01:19:50.297043 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:50.297667 26546 external_mini_cluster.cc:1428] Setting key 5f304ecc1a4dc8f68cf2ddff81b85576
I20260806 01:19:50.299855 27173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:19:50 dist-test-slave-7nm7 krb5kdc[27055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:19:50.317425 27173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2: Bootstrap starting.
I20260806 01:19:50.321110 27173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:50.322363 27173 log.cc:824] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:50.325564 27173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2: No bootstrap required, opened a new log
I20260806 01:19:50.327157 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:50.299784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44045 (local address 127.25.236.190:38901)
0806 01:19:50.300950 (+ 1166us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:50.300971 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:50.301048 (+ 77us) server_negotiation.cc:407] Connection header received
0806 01:19:50.303609 (+ 2561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:50.303646 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:50.304290 (+ 644us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:50.304910 (+ 620us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:50.306971 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:50.308497 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:50.309834 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:50.310692 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:50.310900 (+ 208us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:50.316531 (+ 5631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:50.316540 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:50.316558 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:50.316608 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:50.320966 (+ 4358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:50.322465 (+ 1499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:50.322477 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:50.322509 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:50.322735 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:50.323506 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:50.323511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:50.323522 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:50.324534 (+ 1012us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:50.324957 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:50.325656 (+ 699us) server_negotiation.cc:299] Negotiation successful
0806 01:19:50.325997 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":825,"thread_start_us":480,"threads_started":1}
I20260806 01:19:50.332486 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:50.338734 27173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5887d0642d54434bab4864af4f5457f2" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38901 } }
I20260806 01:19:50.339205 27173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:50.339349 27173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5887d0642d54434bab4864af4f5457f2, State: Initialized, Role: FOLLOWER
I20260806 01:19:50.339993 27173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5887d0642d54434bab4864af4f5457f2" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38901 } }
I20260806 01:19:50.340389 27173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:50.340562 27173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:50.340758 27173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:50.343724 27173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5887d0642d54434bab4864af4f5457f2" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38901 } }
I20260806 01:19:50.344419 27173 leader_election.cc:302] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5887d0642d54434bab4864af4f5457f2; no voters:
I20260806 01:19:50.345139 27173 leader_election.cc:288] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:50.345381 27186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:50.347602 27186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5887d0642d54434bab4864af4f5457f2, State: Running, Role: LEADER
I20260806 01:19:50.348354 27186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5887d0642d54434bab4864af4f5457f2" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38901 } }
I20260806 01:19:50.349510 27173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:50.351259 27189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5887d0642d54434bab4864af4f5457f2. Latest consensus state: current_term: 1 leader_uuid: "5887d0642d54434bab4864af4f5457f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5887d0642d54434bab4864af4f5457f2" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38901 } } }
I20260806 01:19:50.351259 27187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5887d0642d54434bab4864af4f5457f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5887d0642d54434bab4864af4f5457f2" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 38901 } } }
I20260806 01:19:50.351753 27187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:50.351753 27189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:50.364642 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:50.337849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49137 (local address 127.25.236.190:38901)
0806 01:19:50.338362 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:50.338373 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:50.338950 (+ 577us) server_negotiation.cc:407] Connection header received
0806 01:19:50.339260 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:50.339266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:50.339368 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:50.339486 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:50.342164 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:50.343386 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:50.345268 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:50.346507 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:50.347863 (+ 1356us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:50.351677 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:50.351689 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:50.351697 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:50.351763 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:50.358469 (+ 6706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:50.359929 (+ 1460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:50.359939 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:50.359945 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:50.360147 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:50.360878 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:50.360887 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:50.360893 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:50.361423 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:50.361803 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:50.363615 (+ 1812us) server_negotiation.cc:299] Negotiation successful
0806 01:19:50.363861 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 01:19:50.368431 27194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:50.372838 27194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:50.382651 27194 catalog_manager.cc:1347] Generated new cluster ID: e65fcf6347a34cc38e9835f57793df4f
I20260806 01:19:50.382779 27194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:50.392050 27194 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:50.393455 27194 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:50.401998 27194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5887d0642d54434bab4864af4f5457f2: Generated new TSK 0
I20260806 01:19:50.403024 27194 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:50.412029 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27077
I20260806 01:19:50.413875 27163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:50.513260 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 27077
W20260806 01:19:50.537902 27051 connection.cc:535] client connection to 127.25.236.190:38901 recv error: Network error: failed to read from TLS socket (remote: 127.25.236.190:38901): Connection reset by peer (error 104)
2026-08-06T01:19:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2535 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:50 dist-test-slave-7nm7 krb5kdc[27215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:50 dist-test-slave-7nm7 krb5kdc[27215](info): set up 2 sockets
Aug 06 01:19:50 dist-test-slave-7nm7 krb5kdc[27215](info): commencing operation
krb5kdc: starting...
W20260806 01:19:51.912544 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.354s user 0.002s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:52 dist-test-slave-7nm7 krb5kdc[27215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:19:52.293955 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:39821
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:39821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:52.693220 27231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:52.693562 27231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:52.693673 27231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:52.701598 27231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:52.701747 27231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:52.701797 27231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:52.701854 27231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:52.701902 27231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:52.712754 27231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:39821
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:39821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:52.714102 27231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:52.715453 27231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:52.730662 27241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:52.731045 27245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:52.730975 27242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:52.731983 27231 server_base.cc:1034] running on GCE node
I20260806 01:19:52.732810 27231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:52.734470 27231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:52.735708 27231 hybrid_clock.cc:648] HybridClock initialized: now 1785979192735666 us; error 36 us; skew 500 ppm
I20260806 01:19:52.736146 27231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:52 dist-test-slave-7nm7 krb5kdc[27215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979192, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:52.740245 27231 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:19:52.742419 27231 webserver.cc:460] Webserver started at http://127.25.236.190:38079/ using document root <none> and password file <none>
I20260806 01:19:52.743222 27231 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:52.743399 27231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:52.743759 27231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:52.747208 27231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "cc0296391fb04fb4955f6dff3bafcf3e"
format_stamp: "Formatted at 2026-08-06 01:19:52 on dist-test-slave-7nm7"
server_key: "917db2042993f3a4d98340b7cabb3bd5"
server_key_iv: "047687df7dcee877a0dd1bb390be0560"
server_key_version: "encryptionkey@0"
I20260806 01:19:52.748090 27231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "cc0296391fb04fb4955f6dff3bafcf3e"
format_stamp: "Formatted at 2026-08-06 01:19:52 on dist-test-slave-7nm7"
server_key: "917db2042993f3a4d98340b7cabb3bd5"
server_key_iv: "047687df7dcee877a0dd1bb390be0560"
server_key_version: "encryptionkey@0"
I20260806 01:19:52.753989 27231 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260806 01:19:52.758483 27256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:52.760305 27231 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:19:52.760547 27231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "cc0296391fb04fb4955f6dff3bafcf3e"
format_stamp: "Formatted at 2026-08-06 01:19:52 on dist-test-slave-7nm7"
server_key: "917db2042993f3a4d98340b7cabb3bd5"
server_key_iv: "047687df7dcee877a0dd1bb390be0560"
server_key_version: "encryptionkey@0"
I20260806 01:19:52.760785 27231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:52.782416 27231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:52.787395 27231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:52.787734 27231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:52.810300 27231 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:39821
I20260806 01:19:52.810396 27328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:39821 every 8 connection(s)
I20260806 01:19:52.811717 27231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:52.815910 27329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:52.819169 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 27231
I20260806 01:19:52.819383 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:52.819835 26546 external_mini_cluster.cc:1428] Setting key bb57982e03b9d98ef3a96a9de09111ff
I20260806 01:19:52.829836 27329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e: Bootstrap starting.
I20260806 01:19:52.833812 27329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:52.835153 27329 log.cc:824] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:52.835834 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:52.821942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34069 (local address 127.25.236.190:39821)
0806 01:19:52.822879 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:52.822891 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:52.822935 (+ 44us) server_negotiation.cc:407] Connection header received
0806 01:19:52.825221 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:52.825247 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:52.825684 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:52.826047 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:52.827091 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:52.828592 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:52.830549 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:52.831504 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:52.831656 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:52.833163 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:52.833172 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:52.833187 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:52.833750 (+ 563us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:52.833835 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:52.833866 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:52.834086 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:52.834504 (+ 418us) server_negotiation.cc:299] Negotiation successful
0806 01:19:52.834690 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":487,"threads_started":1}
I20260806 01:19:52.838964 27329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e: No bootstrap required, opened a new log
I20260806 01:19:52.841288 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:52.849674 27329 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc0296391fb04fb4955f6dff3bafcf3e" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39821 } }
I20260806 01:19:52.850045 27329 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:52.850199 27329 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0296391fb04fb4955f6dff3bafcf3e, State: Initialized, Role: FOLLOWER
I20260806 01:19:52.850838 27329 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc0296391fb04fb4955f6dff3bafcf3e" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39821 } }
I20260806 01:19:52.851109 27329 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:52.851279 27329 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:52.851450 27329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:52.853040 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:52.846049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53709 (local address 127.25.236.190:39821)
0806 01:19:52.846266 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:52.846273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:52.846660 (+ 387us) server_negotiation.cc:407] Connection header received
0806 01:19:52.846813 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:52.846818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:52.846892 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:52.846994 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:52.847511 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:52.848725 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:52.850197 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:52.850848 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:52.850997 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:52.852245 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:52.852253 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:52.852266 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:52.852285 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:52.852326 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:52.852340 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:52.852479 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:52.852682 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 01:19:52.852739 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:19:52.854359 27329 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc0296391fb04fb4955f6dff3bafcf3e" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39821 } }
I20260806 01:19:52.854967 27329 leader_election.cc:302] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc0296391fb04fb4955f6dff3bafcf3e; no voters:
I20260806 01:19:52.855506 27329 leader_election.cc:288] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:52.855691 27343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:52.856508 27343 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [term 1 LEADER]: Becoming Leader. State: Replica: cc0296391fb04fb4955f6dff3bafcf3e, State: Running, Role: LEADER
I20260806 01:19:52.857050 27343 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc0296391fb04fb4955f6dff3bafcf3e" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39821 } }
I20260806 01:19:52.857697 27329 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:52.860225 27344 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc0296391fb04fb4955f6dff3bafcf3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc0296391fb04fb4955f6dff3bafcf3e" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39821 } } }
I20260806 01:19:52.860791 27344 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:52.860255 27345 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc0296391fb04fb4955f6dff3bafcf3e. Latest consensus state: current_term: 1 leader_uuid: "cc0296391fb04fb4955f6dff3bafcf3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc0296391fb04fb4955f6dff3bafcf3e" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 39821 } } }
I20260806 01:19:52.861678 27345 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:52.869691 27352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:52.876538 27352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:52.888278 27352 catalog_manager.cc:1347] Generated new cluster ID: 58afcca7ce4342c5943340ff502a758e
I20260806 01:19:52.888427 27352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:52.914363 27352 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:52.916787 27352 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:52.929617 27352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc0296391fb04fb4955f6dff3bafcf3e: Generated new TSK 0
I20260806 01:19:52.930639 27352 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:52.947772 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27231
W20260806 01:19:52.949024 27323 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34069
W20260806 01:19:52.949800 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27231: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:19:52.949951 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 27231
W20260806 01:19:52.978904 27213 connection.cc:535] client connection to 127.25.236.190:39821 recv error: Network error: failed to read from TLS socket (remote: 127.25.236.190:39821): Connection reset by peer (error 104)
2026-08-06T01:19:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:53 dist-test-slave-7nm7 krb5kdc[27372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:53 dist-test-slave-7nm7 krb5kdc[27372](info): set up 2 sockets
Aug 06 01:19:53 dist-test-slave-7nm7 krb5kdc[27372](info): commencing operation
krb5kdc: starting...
W20260806 01:19:54.321008 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.000s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:54 dist-test-slave-7nm7 krb5kdc[27372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:19:54.736042 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:37501
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:37501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:55.128031 27387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:55.128347 27387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:55.128444 27387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:55.136575 27387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:55.136703 27387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:55.136755 27387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:55.136821 27387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:55.136868 27387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:55.150007 27387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:37501
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:37501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:55.151547 27387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:55.152855 27387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:55.166716 27392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:55.166716 27393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:55.166770 27399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:55.167574 27387 server_base.cc:1034] running on GCE node
I20260806 01:19:55.168401 27387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:55.169869 27387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:55.171125 27387 hybrid_clock.cc:648] HybridClock initialized: now 1785979195171058 us; error 60 us; skew 500 ppm
I20260806 01:19:55.171505 27387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:55 dist-test-slave-7nm7 krb5kdc[27372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979195, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:55.174876 27387 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:19:55.176549 27387 webserver.cc:460] Webserver started at http://127.25.236.190:38819/ using document root <none> and password file <none>
I20260806 01:19:55.177212 27387 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:55.177321 27387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:55.177579 27387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:55.180773 27387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "306e972dd9c64abda07d48d3d8db07f4"
format_stamp: "Formatted at 2026-08-06 01:19:55 on dist-test-slave-7nm7"
server_key: "3100d77bc2b9cb89f25d10e4858c79bc"
server_key_iv: "001221b5ef458c991d025a31d5dec537"
server_key_version: "encryptionkey@0"
I20260806 01:19:55.181545 27387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "306e972dd9c64abda07d48d3d8db07f4"
format_stamp: "Formatted at 2026-08-06 01:19:55 on dist-test-slave-7nm7"
server_key: "3100d77bc2b9cb89f25d10e4858c79bc"
server_key_iv: "001221b5ef458c991d025a31d5dec537"
server_key_version: "encryptionkey@0"
I20260806 01:19:55.187454 27387 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 01:19:55.191435 27413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:55.192965 27387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:19:55.193130 27387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "306e972dd9c64abda07d48d3d8db07f4"
format_stamp: "Formatted at 2026-08-06 01:19:55 on dist-test-slave-7nm7"
server_key: "3100d77bc2b9cb89f25d10e4858c79bc"
server_key_iv: "001221b5ef458c991d025a31d5dec537"
server_key_version: "encryptionkey@0"
I20260806 01:19:55.193324 27387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:55.199255 27387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:55.204429 27387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:55.204717 27387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:55.225011 27387 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:37501
I20260806 01:19:55.225052 27487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:37501 every 8 connection(s)
I20260806 01:19:55.226646 27387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:55.230710 27488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:19:55.236023 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 27387
I20260806 01:19:55.236238 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:55.237401 26546 external_mini_cluster.cc:1428] Setting key 1b2afd51e893e1a3d8773aceafa65396
I20260806 01:19:55.244249 27488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4: Bootstrap starting.
I20260806 01:19:55.247102 27492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:55.240137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57471 (local address 127.25.236.190:37501)
0806 01:19:55.241030 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:55.241042 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:55.241100 (+ 58us) server_negotiation.cc:407] Connection header received
0806 01:19:55.243080 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:55.243105 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:55.243462 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:55.243814 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:55.245367 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:55.245374 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:55.245386 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:55.245403 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:55.245474 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:55.245496 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:55.245680 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:55.245834 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 01:19:55.246063 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":422,"threads_started":1}
I20260806 01:19:55.248237 27488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:55.249331 27488 log.cc:824] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:55.251173 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:55.253072 27488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4: No bootstrap required, opened a new log
I20260806 01:19:55.262144 27492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:55.258412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40909 (local address 127.25.236.190:37501)
0806 01:19:55.258621 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:55.258625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:55.258979 (+ 354us) server_negotiation.cc:407] Connection header received
0806 01:19:55.259367 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:55.259372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:55.259426 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:55.259543 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:55.261184 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:55.261188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:55.261192 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:19:55.261208 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:55.261250 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:19:55.261267 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:55.261388 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:55.261709 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 01:19:55.261764 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 01:19:55.266474 27488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "306e972dd9c64abda07d48d3d8db07f4" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37501 } }
I20260806 01:19:55.266913 27488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:55.267036 27488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 306e972dd9c64abda07d48d3d8db07f4, State: Initialized, Role: FOLLOWER
I20260806 01:19:55.267661 27488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "306e972dd9c64abda07d48d3d8db07f4" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37501 } }
I20260806 01:19:55.267915 27488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:55.268090 27488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:55.268271 27488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:55.270782 27488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "306e972dd9c64abda07d48d3d8db07f4" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37501 } }
I20260806 01:19:55.271309 27488 leader_election.cc:302] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 306e972dd9c64abda07d48d3d8db07f4; no voters:
I20260806 01:19:55.271965 27488 leader_election.cc:288] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:55.272197 27504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:55.272929 27504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [term 1 LEADER]: Becoming Leader. State: Replica: 306e972dd9c64abda07d48d3d8db07f4, State: Running, Role: LEADER
I20260806 01:19:55.273667 27504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "306e972dd9c64abda07d48d3d8db07f4" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37501 } }
I20260806 01:19:55.274461 27488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:55.277062 27506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "306e972dd9c64abda07d48d3d8db07f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "306e972dd9c64abda07d48d3d8db07f4" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37501 } } }
I20260806 01:19:55.277367 27506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:55.276940 27508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 306e972dd9c64abda07d48d3d8db07f4. Latest consensus state: current_term: 1 leader_uuid: "306e972dd9c64abda07d48d3d8db07f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "306e972dd9c64abda07d48d3d8db07f4" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37501 } } }
I20260806 01:19:55.277851 27508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4 [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:55.280222 27518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:55.285898 27518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:55.296267 27518 catalog_manager.cc:1347] Generated new cluster ID: 708d463fa71e44a5b148521bdf5d0804
I20260806 01:19:55.296406 27518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:55.311021 27518 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:55.311477 27518 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:55.329684 27518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 306e972dd9c64abda07d48d3d8db07f4: Generated new TSK 0
I20260806 01:19:55.330804 27518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:55.336409 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27387
W20260806 01:19:55.338444 27481 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57471
W20260806 01:19:55.339252 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27387: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:19:55.339465 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 27387
2026-08-06T01:19:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2388 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:55 dist-test-slave-7nm7 krb5kdc[27533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:55 dist-test-slave-7nm7 krb5kdc[27533](info): set up 2 sockets
Aug 06 01:19:55 dist-test-slave-7nm7 krb5kdc[27533](info): commencing operation
krb5kdc: starting...
W20260806 01:19:56.783344 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.396s user 0.003s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:57 dist-test-slave-7nm7 krb5kdc[27533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:19:57.171054 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:41783
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:40649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:41783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:19:57.578943 27555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:19:57.579360 27555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:19:57.579543 27555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:19:57.589512 27555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:19:57.589653 27555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:19:57.589702 27555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:19:57.589758 27555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:19:57.589812 27555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:19:57.600785 27555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:40649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:41783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:41783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:19:57.602818 27555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:19:57.604547 27555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:19:57.618371 27563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:57.618572 27565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:19:57.618551 27567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:19:57.619674 27555 server_base.cc:1034] running on GCE node
I20260806 01:19:57.620759 27555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:19:57.622495 27555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:19:57.623759 27555 hybrid_clock.cc:648] HybridClock initialized: now 1785979197623717 us; error 61 us; skew 500 ppm
I20260806 01:19:57.624317 27555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:19:57 dist-test-slave-7nm7 krb5kdc[27533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979197, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:19:57.627993 27555 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:19:57.629760 27555 webserver.cc:460] Webserver started at http://127.25.236.190:38425/ using document root <none> and password file <none>
I20260806 01:19:57.630587 27555 fs_manager.cc:359] Metadata directory not provided
I20260806 01:19:57.630753 27555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:19:57.631094 27555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:19:57.634404 27555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "0a0093479368459da38c9e9b0d64e52c"
format_stamp: "Formatted at 2026-08-06 01:19:57 on dist-test-slave-7nm7"
server_key: "2b7c76785c2a06c275b0c29daaacb198"
server_key_iv: "24f8e86a96fc3596549eaf5d728ba7d2"
server_key_version: "encryptionkey@0"
I20260806 01:19:57.635412 27555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "0a0093479368459da38c9e9b0d64e52c"
format_stamp: "Formatted at 2026-08-06 01:19:57 on dist-test-slave-7nm7"
server_key: "2b7c76785c2a06c275b0c29daaacb198"
server_key_iv: "24f8e86a96fc3596549eaf5d728ba7d2"
server_key_version: "encryptionkey@0"
I20260806 01:19:57.642902 27555 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 01:19:57.647723 27575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:19:57.649300 27555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260806 01:19:57.649529 27555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "0a0093479368459da38c9e9b0d64e52c"
format_stamp: "Formatted at 2026-08-06 01:19:57 on dist-test-slave-7nm7"
server_key: "2b7c76785c2a06c275b0c29daaacb198"
server_key_iv: "24f8e86a96fc3596549eaf5d728ba7d2"
server_key_version: "encryptionkey@0"
I20260806 01:19:57.649762 27555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:19:57.663856 27555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:19:57.669188 27555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:19:57.669503 27555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:19:57.690948 27555 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:41783
I20260806 01:19:57.690984 27652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:41783 every 8 connection(s)
I20260806 01:19:57.692394 27555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:19:57.694074 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 27555
I20260806 01:19:57.694275 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:19:57.694728 26546 external_mini_cluster.cc:1428] Setting key 01565c5276002ce85f9ae8b780869bb2
I20260806 01:19:57.696774 27654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:19:57 dist-test-slave-7nm7 krb5kdc[27533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:19:57.712597 27654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c: Bootstrap starting.
I20260806 01:19:57.716347 27654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c: Neither blocks nor log segments found. Creating new log.
I20260806 01:19:57.717446 27654 log.cc:824] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:19:57.720557 27654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c: No bootstrap required, opened a new log
I20260806 01:19:57.721555 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:57.696888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43041 (local address 127.25.236.190:41783)
0806 01:19:57.698287 (+ 1399us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:57.698306 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:57.698357 (+ 51us) server_negotiation.cc:407] Connection header received
0806 01:19:57.700734 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:57.700756 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:57.701175 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:57.701505 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:57.703561 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:57.704970 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:57.706734 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:57.707386 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:57.707522 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:57.712405 (+ 4883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:57.712414 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:57.712428 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:57.712465 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:57.717305 (+ 4840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:57.718100 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:57.718107 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:57.718113 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:57.718373 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:57.718779 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:57.718785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:57.718788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:57.719325 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:57.719645 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:57.720101 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 01:19:57.720547 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1099,"thread_start_us":815,"threads_started":1}
I20260806 01:19:57.725317 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:19:57.728819 27654 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0093479368459da38c9e9b0d64e52c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 41783 } }
I20260806 01:19:57.729211 27654 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:19:57.729332 27654 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0093479368459da38c9e9b0d64e52c, State: Initialized, Role: FOLLOWER
I20260806 01:19:57.730093 27654 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0093479368459da38c9e9b0d64e52c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 41783 } }
I20260806 01:19:57.730420 27654 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:19:57.730600 27654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:19:57.730813 27654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:19:57.733573 27654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0093479368459da38c9e9b0d64e52c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 41783 } }
I20260806 01:19:57.734124 27654 leader_election.cc:302] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a0093479368459da38c9e9b0d64e52c; no voters:
I20260806 01:19:57.734777 27654 leader_election.cc:288] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:19:57.734941 27670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:19:57.735646 27670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [term 1 LEADER]: Becoming Leader. State: Replica: 0a0093479368459da38c9e9b0d64e52c, State: Running, Role: LEADER
I20260806 01:19:57.736320 27670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0093479368459da38c9e9b0d64e52c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 41783 } }
I20260806 01:19:57.739351 27672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a0093479368459da38c9e9b0d64e52c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0093479368459da38c9e9b0d64e52c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 41783 } } }
I20260806 01:19:57.740096 27672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:57.739887 27670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a0093479368459da38c9e9b0d64e52c. Latest consensus state: current_term: 1 leader_uuid: "0a0093479368459da38c9e9b0d64e52c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0093479368459da38c9e9b0d64e52c" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 41783 } } }
I20260806 01:19:57.740676 27670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [sys.catalog]: This master's current role is: LEADER
I20260806 01:19:57.743072 27654 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:19:57.743280 27676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:19:57.748546 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:19:57.729764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52489 (local address 127.25.236.190:41783)
0806 01:19:57.729949 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 01:19:57.729957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:19:57.730431 (+ 474us) server_negotiation.cc:407] Connection header received
0806 01:19:57.730639 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:19:57.730646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:19:57.730730 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:19:57.730852 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:19:57.732264 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:57.733369 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:57.734988 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:19:57.735804 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:19:57.735927 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:19:57.739583 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:19:57.739593 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:19:57.739598 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:19:57.739646 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:19:57.743614 (+ 3968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:57.744485 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:57.744491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:57.744494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:57.744867 (+ 373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:19:57.745422 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:19:57.745434 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:19:57.745440 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:19:57.745985 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:19:57.746396 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0806 01:19:57.748115 (+ 1719us) server_negotiation.cc:299] Negotiation successful
0806 01:19:57.748293 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 01:19:57.749661 27676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:19:57.758821 27676 catalog_manager.cc:1347] Generated new cluster ID: 714674cfcaab4675ab5e4d912cab7e03
I20260806 01:19:57.758926 27676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:19:57.768683 27676 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:19:57.770059 27676 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:19:57.777628 27676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a0093479368459da38c9e9b0d64e52c: Generated new TSK 0
I20260806 01:19:57.778571 27676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:19:57.790158 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27555
I20260806 01:19:57.791819 27647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:19:57.886901 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 27555
2026-08-06T01:19:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:19:58 dist-test-slave-7nm7 krb5kdc[27705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:19:58 dist-test-slave-7nm7 krb5kdc[27705](info): set up 2 sockets
Aug 06 01:19:58 dist-test-slave-7nm7 krb5kdc[27705](info): commencing operation
krb5kdc: starting...
W20260806 01:19:59.282568 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.346s user 0.000s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:19:59 dist-test-slave-7nm7 krb5kdc[27705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:19:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:19:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:19:59.649588 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:45287
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:46725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:45287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:20:00.016992 27722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:20:00.017323 27722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:20:00.017426 27722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:20:00.027240 27722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:20:00.027383 27722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:20:00.027441 27722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:20:00.027498 27722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:20:00.027546 27722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:20:00.038332 27722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:46725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:45287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:45287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:20:00.039680 27722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:20:00.041009 27722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:20:00.053568 27734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:00.053644 27727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:00.053676 27731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:20:00.054569 27722 server_base.cc:1034] running on GCE node
I20260806 01:20:00.055406 27722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:20:00.056901 27722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:20:00.058140 27722 hybrid_clock.cc:648] HybridClock initialized: now 1785979200058058 us; error 55 us; skew 500 ppm
I20260806 01:20:00.058576 27722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:20:00 dist-test-slave-7nm7 krb5kdc[27705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979200, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:20:00.062884 27722 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:20:00.064589 27722 webserver.cc:460] Webserver started at http://127.25.236.190:36815/ using document root <none> and password file <none>
I20260806 01:20:00.065274 27722 fs_manager.cc:359] Metadata directory not provided
I20260806 01:20:00.065387 27722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:20:00.065639 27722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:20:00.069123 27722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "e093545dd78a49f0b25a8b46b268351e"
format_stamp: "Formatted at 2026-08-06 01:20:00 on dist-test-slave-7nm7"
server_key: "443988d511c5ee57369c3504e7ba3d66"
server_key_iv: "304b1db90ced3b11254356e0b08c2c06"
server_key_version: "encryptionkey@0"
I20260806 01:20:00.069880 27722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "e093545dd78a49f0b25a8b46b268351e"
format_stamp: "Formatted at 2026-08-06 01:20:00 on dist-test-slave-7nm7"
server_key: "443988d511c5ee57369c3504e7ba3d66"
server_key_iv: "304b1db90ced3b11254356e0b08c2c06"
server_key_version: "encryptionkey@0"
I20260806 01:20:00.075489 27722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 01:20:00.079285 27742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:20:00.080734 27722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:20:00.080936 27722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "e093545dd78a49f0b25a8b46b268351e"
format_stamp: "Formatted at 2026-08-06 01:20:00 on dist-test-slave-7nm7"
server_key: "443988d511c5ee57369c3504e7ba3d66"
server_key_iv: "304b1db90ced3b11254356e0b08c2c06"
server_key_version: "encryptionkey@0"
I20260806 01:20:00.081148 27722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:20:00.107262 27722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:20:00.111779 27722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:20:00.112192 27722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:20:00.134145 27722 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:45287
I20260806 01:20:00.134178 27818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:45287 every 8 connection(s)
I20260806 01:20:00.135879 27722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:20:00.137600 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 27722
I20260806 01:20:00.137812 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:20:00.138603 26546 external_mini_cluster.cc:1428] Setting key 6e13a2ff3befc47d1cb61f2ecd90174c
I20260806 01:20:00.140179 27819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:20:00.152856 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:00.140582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58679 (local address 127.25.236.190:45287)
0806 01:20:00.141702 (+ 1120us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:00.141715 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:00.141762 (+ 47us) server_negotiation.cc:407] Connection header received
0806 01:20:00.143634 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:00.143657 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:00.144052 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:00.144490 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:00.145230 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:00.146667 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:00.148037 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:00.148850 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:00.149071 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:00.150414 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:00.150421 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:00.150436 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:20:00.151087 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:00.151172 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:20:00.151199 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:00.151348 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:00.151596 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 01:20:00.151880 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":894,"thread_start_us":495,"threads_started":1}
I20260806 01:20:00.153762 27819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e: Bootstrap starting.
I20260806 01:20:00.157657 27819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e: Neither blocks nor log segments found. Creating new log.
I20260806 01:20:00.157673 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:20:00.158787 27819 log.cc:824] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:20:00.161793 27819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e: No bootstrap required, opened a new log
I20260806 01:20:00.169611 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:00.162327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44109 (local address 127.25.236.190:45287)
0806 01:20:00.162517 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:00.162525 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:00.162931 (+ 406us) server_negotiation.cc:407] Connection header received
0806 01:20:00.163159 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:00.163165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:00.163242 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:00.163348 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:00.163894 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:00.165047 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:00.166877 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:00.167404 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:00.167576 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:00.168842 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:00.168846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:00.168850 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:20:00.168872 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:00.168912 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:20:00.168927 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:00.169052 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:00.169194 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 01:20:00.169290 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 01:20:00.171280 27819 raft_consensus.cc:348] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e093545dd78a49f0b25a8b46b268351e" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 45287 } }
I20260806 01:20:00.171720 27819 raft_consensus.cc:374] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:20:00.171841 27819 raft_consensus.cc:729] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e093545dd78a49f0b25a8b46b268351e, State: Initialized, Role: FOLLOWER
I20260806 01:20:00.172413 27819 consensus_queue.cc:260] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e093545dd78a49f0b25a8b46b268351e" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 45287 } }
I20260806 01:20:00.172672 27819 raft_consensus.cc:388] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:20:00.172825 27819 raft_consensus.cc:482] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:20:00.173014 27819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:20:00.175805 27819 raft_consensus.cc:504] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e093545dd78a49f0b25a8b46b268351e" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 45287 } }
I20260806 01:20:00.176342 27819 leader_election.cc:302] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e093545dd78a49f0b25a8b46b268351e; no voters:
I20260806 01:20:00.177071 27819 leader_election.cc:288] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:20:00.177208 27838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:20:00.178248 27838 raft_consensus.cc:686] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [term 1 LEADER]: Becoming Leader. State: Replica: e093545dd78a49f0b25a8b46b268351e, State: Running, Role: LEADER
I20260806 01:20:00.178845 27838 consensus_queue.cc:237] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e093545dd78a49f0b25a8b46b268351e" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 45287 } }
I20260806 01:20:00.179819 27819 sys_catalog.cc:564] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:20:00.181790 27840 sys_catalog.cc:455] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e093545dd78a49f0b25a8b46b268351e. Latest consensus state: current_term: 1 leader_uuid: "e093545dd78a49f0b25a8b46b268351e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e093545dd78a49f0b25a8b46b268351e" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 45287 } } }
I20260806 01:20:00.182219 27840 sys_catalog.cc:458] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [sys.catalog]: This master's current role is: LEADER
I20260806 01:20:00.182750 27839 sys_catalog.cc:455] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e093545dd78a49f0b25a8b46b268351e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e093545dd78a49f0b25a8b46b268351e" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 45287 } } }
I20260806 01:20:00.183138 27839 sys_catalog.cc:458] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e [sys.catalog]: This master's current role is: LEADER
I20260806 01:20:00.185789 27846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:20:00.191210 27846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:20:00.201190 27846 catalog_manager.cc:1347] Generated new cluster ID: 5f675dbbaa21476aa62ae4bc9efdaa83
I20260806 01:20:00.201326 27846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:20:00.245602 27846 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:20:00.247303 27846 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:20:00.266502 27846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e093545dd78a49f0b25a8b46b268351e: Generated new TSK 0
I20260806 01:20:00.267426 27846 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:20:00.279068 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27722
W20260806 01:20:00.280443 27810 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58679
W20260806 01:20:00.281351 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27722: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:20:00.281682 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 27722
2026-08-06T01:20:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:20:00 dist-test-slave-7nm7 krb5kdc[27870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:20:00 dist-test-slave-7nm7 krb5kdc[27870](info): set up 2 sockets
Aug 06 01:20:00 dist-test-slave-7nm7 krb5kdc[27870](info): commencing operation
krb5kdc: starting...
W20260806 01:20:01.679194 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.350s user 0.001s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:20:01 dist-test-slave-7nm7 krb5kdc[27870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:20:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:20:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:20:02.091053 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:37277
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:37277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:20:02.492156 27887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:20:02.492558 27887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:20:02.492728 27887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:20:02.501237 27887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:20:02.501454 27887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:20:02.501569 27887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:20:02.501683 27887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:20:02.501775 27887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:20:02.512977 27887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:37277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:37277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:20:02.514461 27887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:20:02.515846 27887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:20:02.531164 27899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:02.531261 27898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:02.531237 27901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:20:02.532668 27887 server_base.cc:1034] running on GCE node
I20260806 01:20:02.534044 27887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:20:02.536054 27887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:20:02.537402 27887 hybrid_clock.cc:648] HybridClock initialized: now 1785979202537363 us; error 48 us; skew 500 ppm
I20260806 01:20:02.537837 27887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:20:02 dist-test-slave-7nm7 krb5kdc[27870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979202, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:20:02.547008 27887 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:20:02.548707 27887 webserver.cc:460] Webserver started at http://127.25.236.190:38775/ using document root <none> and password file <none>
I20260806 01:20:02.549425 27887 fs_manager.cc:359] Metadata directory not provided
I20260806 01:20:02.549536 27887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:20:02.549826 27887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:20:02.553634 27887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "cf826f0dda0d4b13be78fcd788fc9244"
format_stamp: "Formatted at 2026-08-06 01:20:02 on dist-test-slave-7nm7"
server_key: "530d56da73f9099d76bb6b7ba02af3e5"
server_key_iv: "bbba404e1a684fe23c584004d7e9effd"
server_key_version: "encryptionkey@0"
I20260806 01:20:02.555186 27887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "cf826f0dda0d4b13be78fcd788fc9244"
format_stamp: "Formatted at 2026-08-06 01:20:02 on dist-test-slave-7nm7"
server_key: "530d56da73f9099d76bb6b7ba02af3e5"
server_key_iv: "bbba404e1a684fe23c584004d7e9effd"
server_key_version: "encryptionkey@0"
I20260806 01:20:02.561625 27887 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 01:20:02.565894 27912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:20:02.567616 27887 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:20:02.567886 27887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "cf826f0dda0d4b13be78fcd788fc9244"
format_stamp: "Formatted at 2026-08-06 01:20:02 on dist-test-slave-7nm7"
server_key: "530d56da73f9099d76bb6b7ba02af3e5"
server_key_iv: "bbba404e1a684fe23c584004d7e9effd"
server_key_version: "encryptionkey@0"
I20260806 01:20:02.568106 27887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:20:02.574582 27887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:20:02.579934 27887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:20:02.580240 27887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:20:02.604579 27887 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:37277
I20260806 01:20:02.604602 27983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:37277 every 8 connection(s)
I20260806 01:20:02.606135 27887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:20:02.611689 27984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:20:02.614749 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 27887
I20260806 01:20:02.615082 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:20:02.615777 26546 external_mini_cluster.cc:1428] Setting key 79277cf059d323b75c9141518a00d9cf
I20260806 01:20:02.626777 27990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:02.618108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55605 (local address 127.25.236.190:37277)
0806 01:20:02.619252 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:02.619274 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:02.619339 (+ 65us) server_negotiation.cc:407] Connection header received
0806 01:20:02.621764 (+ 2425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:02.621801 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:02.622459 (+ 658us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:02.622944 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:02.624643 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:02.624653 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:02.624671 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:20:02.624697 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:02.624792 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:20:02.624829 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:02.625079 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:02.625243 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 01:20:02.625461 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":864,"thread_start_us":454,"threads_started":1}
I20260806 01:20:02.627558 27984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244: Bootstrap starting.
I20260806 01:20:02.631743 27984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244: Neither blocks nor log segments found. Creating new log.
I20260806 01:20:02.632303 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:20:02.632956 27984 log.cc:824] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244: Log is configured to *not* fsync() on all Append() calls
I20260806 01:20:02.636307 27984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244: No bootstrap required, opened a new log
I20260806 01:20:02.641894 27990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:02.638560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47959 (local address 127.25.236.190:37277)
0806 01:20:02.638835 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:02.638841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:02.639454 (+ 613us) server_negotiation.cc:407] Connection header received
0806 01:20:02.639698 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:02.639703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:02.639761 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:02.639857 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:02.641116 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:02.641120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:02.641124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:20:02.641143 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:02.641183 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:20:02.641198 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:02.641300 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:02.641455 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 01:20:02.641516 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 01:20:02.646029 27984 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf826f0dda0d4b13be78fcd788fc9244" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37277 } }
I20260806 01:20:02.646551 27984 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:20:02.646687 27984 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf826f0dda0d4b13be78fcd788fc9244, State: Initialized, Role: FOLLOWER
I20260806 01:20:02.647434 27984 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf826f0dda0d4b13be78fcd788fc9244" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37277 } }
I20260806 01:20:02.647661 27984 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:20:02.647768 27984 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:20:02.647902 27984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:20:02.650087 27984 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf826f0dda0d4b13be78fcd788fc9244" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37277 } }
I20260806 01:20:02.650653 27984 leader_election.cc:302] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf826f0dda0d4b13be78fcd788fc9244; no voters:
I20260806 01:20:02.651345 27984 leader_election.cc:288] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:20:02.651562 28001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:20:02.652290 28001 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [term 1 LEADER]: Becoming Leader. State: Replica: cf826f0dda0d4b13be78fcd788fc9244, State: Running, Role: LEADER
I20260806 01:20:02.652915 28001 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf826f0dda0d4b13be78fcd788fc9244" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37277 } }
I20260806 01:20:02.653649 27984 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:20:02.656113 28005 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf826f0dda0d4b13be78fcd788fc9244. Latest consensus state: current_term: 1 leader_uuid: "cf826f0dda0d4b13be78fcd788fc9244" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf826f0dda0d4b13be78fcd788fc9244" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37277 } } }
I20260806 01:20:02.656610 28005 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [sys.catalog]: This master's current role is: LEADER
I20260806 01:20:02.657059 28003 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf826f0dda0d4b13be78fcd788fc9244" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf826f0dda0d4b13be78fcd788fc9244" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 37277 } } }
I20260806 01:20:02.658038 28003 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244 [sys.catalog]: This master's current role is: LEADER
I20260806 01:20:02.658375 28017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:20:02.664968 28017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:20:02.675665 28017 catalog_manager.cc:1347] Generated new cluster ID: d53c0e2190c0442d9ba20318aa7900b4
I20260806 01:20:02.675808 28017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:20:02.698453 28017 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:20:02.698937 28017 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:20:02.717941 28017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf826f0dda0d4b13be78fcd788fc9244: Generated new TSK 0
I20260806 01:20:02.719089 28017 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:20:02.735476 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27887
W20260806 01:20:02.737571 27978 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55605
W20260806 01:20:02.738528 26546 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 27887: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:20:02.738688 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 27887
2026-08-06T01:20:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2457 ms)
[----------] 18 tests from AuthTokenIssuingTest (43760 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:20:02 dist-test-slave-7nm7 krb5kdc[28033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:20:02 dist-test-slave-7nm7 krb5kdc[28033](info): set up 2 sockets
Aug 06 01:20:02 dist-test-slave-7nm7 krb5kdc[28033](info): commencing operation
krb5kdc: starting...
W20260806 01:20:04.181769 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.394s user 0.003s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:20:04 dist-test-slave-7nm7 krb5kdc[28033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:20:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:20:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:20:04.595625 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:46847
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:34119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:46847,127.25.236.189:38957,127.25.236.188:43349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:20:04.986375 28052 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:20:04.986913 28052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:20:04.987085 28052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:20:04.987244 28052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:20:04.995476 28052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:20:04.995620 28052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:20:04.995733 28052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:20:04.995837 28052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:20:04.995932 28052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:20:05.006525 28052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:34119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:46847,127.25.236.189:38957,127.25.236.188:43349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:46847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:20:05.007912 28052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:20:05.009260 28052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:20:05.021514 28062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:05.021602 28060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:05.021638 28065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:20:05.022459 28052 server_base.cc:1034] running on GCE node
I20260806 01:20:05.023272 28052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:20:05.024593 28052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:20:05.025827 28052 hybrid_clock.cc:648] HybridClock initialized: now 1785979205025768 us; error 28 us; skew 500 ppm
I20260806 01:20:05.026162 28052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:20:05 dist-test-slave-7nm7 krb5kdc[28033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979205, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:20:05.029417 28052 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:20:05.030900 28052 webserver.cc:460] Webserver started at http://127.25.236.190:36631/ using document root <none> and password file <none>
I20260806 01:20:05.031577 28052 fs_manager.cc:359] Metadata directory not provided
I20260806 01:20:05.031677 28052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:20:05.031950 28052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:20:05.035006 28052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "1549bca01c084175848f513ec63b3fc8"
format_stamp: "Formatted at 2026-08-06 01:20:05 on dist-test-slave-7nm7"
server_key: "09b04c98bce306dbd590b314c02b67ba"
server_key_iv: "86924a118a05e92ac1e2c00acc193ded"
server_key_version: "encryptionkey@0"
I20260806 01:20:05.035894 28052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "1549bca01c084175848f513ec63b3fc8"
format_stamp: "Formatted at 2026-08-06 01:20:05 on dist-test-slave-7nm7"
server_key: "09b04c98bce306dbd590b314c02b67ba"
server_key_iv: "86924a118a05e92ac1e2c00acc193ded"
server_key_version: "encryptionkey@0"
I20260806 01:20:05.041752 28052 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 01:20:05.046167 28079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:20:05.047914 28052 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260806 01:20:05.048134 28052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "1549bca01c084175848f513ec63b3fc8"
format_stamp: "Formatted at 2026-08-06 01:20:05 on dist-test-slave-7nm7"
server_key: "09b04c98bce306dbd590b314c02b67ba"
server_key_iv: "86924a118a05e92ac1e2c00acc193ded"
server_key_version: "encryptionkey@0"
I20260806 01:20:05.048413 28052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:20:05.072387 28052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:20:05.077054 28052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:20:05.077561 28052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:20:05.101969 28052 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:46847
I20260806 01:20:05.101994 28148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:46847 every 8 connection(s)
I20260806 01:20:05.103451 28052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:20:05.106961 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 28052
I20260806 01:20:05.107180 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:20:05.107616 28149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:20:05.107903 26546 external_mini_cluster.cc:1428] Setting key 239a66b296c92cf1ffba993eea014d90
I20260806 01:20:05.114773 28149 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:20:05 dist-test-slave-7nm7 krb5kdc[28033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979205, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:20:05.142915 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.118341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:46847 (local address 127.0.0.1:37426)
0806 01:20:05.119623 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.119678 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:05.123109 (+ 3431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:05.124077 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:05.124087 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:05.124919 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:05.125270 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:05.125425 (+ 155us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.127489 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.127496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:05.129027 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:05.129057 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.130696 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.130705 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:05.130915 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:05.131624 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:05.131654 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:05.133871 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:05.137478 (+ 3607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:05.137494 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:05.137509 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:05.137938 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:05.138465 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:05.138469 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:05.138473 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:05.138643 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:05.139833 (+ 1190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:05.139842 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:05.140076 (+ 234us) client_negotiation.cc:769] Sending connection context
0806 01:20:05.140498 (+ 422us) client_negotiation.cc:240] Negotiation successful
0806 01:20:05.141514 (+ 1016us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":191,"thread_start_us":443,"threads_started":1}
I20260806 01:20:05.142948 28154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.118341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37426 (local address 127.25.236.190:46847)
0806 01:20:05.120350 (+ 2009us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:05.120356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:05.120402 (+ 46us) server_negotiation.cc:407] Connection header received
0806 01:20:05.123569 (+ 3167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:05.123576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:05.123820 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:05.123995 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:05.125805 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.127231 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.129288 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.130391 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.130618 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:05.134119 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:05.134128 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:05.134141 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:05.134185 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:05.137060 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:05.138124 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:05.138131 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:05.138133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:05.138342 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:05.138769 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:05.138774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:05.138777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:05.139652 (+ 875us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:05.139910 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:05.140624 (+ 714us) server_negotiation.cc:299] Negotiation successful
0806 01:20:05.141057 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":1816,"spinlock_wait_cycles":17280,"thread_start_us":1499,"threads_started":1}
WARNING: no policy specified for kudu/127.25.236.189@KRBTEST.COM; defaulting to no policy
I20260806 01:20:05.146773 28149 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.189@KRBTEST.COM" created.
I20260806 01:20:05.148218 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.147679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:38957 (local address 127.0.0.1:58220)
0806 01:20:05.147905 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.148049 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260806 01:20:05.148598 28085 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
I20260806 01:20:05.150399 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.149905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:38957 (local address 127.0.0.1:58234)
0806 01:20:05.150093 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.150190 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 01:20:05.150772 28149 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:38957: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.189.keytab.
Entry for principal kudu/127.25.236.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.189.keytab.
I20260806 01:20:05.189673 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.189:38957
--webserver_interface=127.25.236.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.189.keytab
--principal=kudu/127.25.236.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:34119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:46847,127.25.236.189:38957,127.25.236.188:43349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:20:05.200156 28149 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } attempt: 1
I20260806 01:20:05.201745 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.200872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:38957 (local address 127.0.0.1:58236)
0806 01:20:05.201273 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.201548 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260806 01:20:05.203056 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.202608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:38957 (local address 127.0.0.1:58246)
0806 01:20:05.202784 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.202878 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 01:20:05.203334 28149 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:38957: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
I20260806 01:20:05.271636 28149 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } attempt: 2
I20260806 01:20:05.272941 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.272447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:38957 (local address 127.0.0.1:58248)
0806 01:20:05.272703 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.272791 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 01:20:05.273959 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.273600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:38957 (local address 127.0.0.1:58256)
0806 01:20:05.273743 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.273830 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 01:20:05.274210 28149 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:38957: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
I20260806 01:20:05.365543 28149 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } attempt: 3
I20260806 01:20:05.366837 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.366381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:38957 (local address 127.0.0.1:58264)
0806 01:20:05.366588 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.366687 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 01:20:05.367848 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.367485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:38957 (local address 127.0.0.1:58278)
0806 01:20:05.367636 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.367706 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 01:20:05.368052 28149 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:38957: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
I20260806 01:20:05.511354 28149 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } attempt: 4
I20260806 01:20:05.512655 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.512069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:38957 (local address 127.0.0.1:58294)
0806 01:20:05.512416 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.512513 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260806 01:20:05.513680 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.513328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:38957 (local address 127.0.0.1:58308)
0806 01:20:05.513472 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.513559 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 01:20:05.513909 28149 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:38957: Network error: Client connection negotiation failed: client connection to 127.25.236.189:38957: connect: Connection refused (error 111)
W20260806 01:20:05.560789 28160 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:20:05.561426 28160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:20:05.561602 28160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:20:05.561793 28160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:20:05.570437 28160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:20:05.570545 28160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:20:05.570643 28160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:20:05.570737 28160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:20:05.570855 28160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:20:05.581485 28160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:34119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:46847,127.25.236.189:38957,127.25.236.188:43349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.189.keytab
--principal=kudu/127.25.236.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.189:38957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.236.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:20:05.583113 28160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:20:05.584476 28160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:20:05.597558 28172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:05.597585 28169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:05.597496 28167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:20:05.598968 28160 server_base.cc:1034] running on GCE node
I20260806 01:20:05.599902 28160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:20:05.601444 28160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:20:05.602743 28160 hybrid_clock.cc:648] HybridClock initialized: now 1785979205602678 us; error 38 us; skew 500 ppm
I20260806 01:20:05.603112 28160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:20:05 dist-test-slave-7nm7 krb5kdc[28033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979205, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:20:05.606750 28160 init.cc:377] Logged in from keytab as kudu/127.25.236.189@KRBTEST.COM (short username kudu)
I20260806 01:20:05.608603 28160 webserver.cc:460] Webserver started at http://127.25.236.189:38537/ using document root <none> and password file <none>
I20260806 01:20:05.609364 28160 fs_manager.cc:359] Metadata directory not provided
I20260806 01:20:05.609508 28160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:20:05.609853 28160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:20:05.613003 28160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/data/instance:
uuid: "e5e12ad72fc04103977df47eb484410d"
format_stamp: "Formatted at 2026-08-06 01:20:05 on dist-test-slave-7nm7"
server_key: "37ab2d56421cab9b6a910f0a73d095f5"
server_key_iv: "ce912fcf07f89e9a4a82b31f84ba97f5"
server_key_version: "encryptionkey@0"
I20260806 01:20:05.613838 28160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/wal/instance:
uuid: "e5e12ad72fc04103977df47eb484410d"
format_stamp: "Formatted at 2026-08-06 01:20:05 on dist-test-slave-7nm7"
server_key: "37ab2d56421cab9b6a910f0a73d095f5"
server_key_iv: "ce912fcf07f89e9a4a82b31f84ba97f5"
server_key_version: "encryptionkey@0"
I20260806 01:20:05.619406 28160 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 01:20:05.623867 28183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:20:05.625385 28160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260806 01:20:05.625607 28160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/wal
uuid: "e5e12ad72fc04103977df47eb484410d"
format_stamp: "Formatted at 2026-08-06 01:20:05 on dist-test-slave-7nm7"
server_key: "37ab2d56421cab9b6a910f0a73d095f5"
server_key_iv: "ce912fcf07f89e9a4a82b31f84ba97f5"
server_key_version: "encryptionkey@0"
I20260806 01:20:05.625869 28160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:20:05.647280 28160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:20:05.652002 28160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:20:05.652359 28160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:20:05.673921 28160 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.189:38957
I20260806 01:20:05.673935 28253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.189:38957 every 8 connection(s)
I20260806 01:20:05.675372 28160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/data/info.pb
I20260806 01:20:05.679258 28255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:20:05.682348 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 28160
I20260806 01:20:05.682559 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-1/wal/instance
I20260806 01:20:05.683111 26546 external_mini_cluster.cc:1428] Setting key 1d81077c683681b140bb252059fabfdf
I20260806 01:20:05.690940 28255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:20:05 dist-test-slave-7nm7 krb5kdc[28033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979205, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.189@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:20:05.721731 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.695430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37434 (local address 127.25.236.190:46847)
0806 01:20:05.696001 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:05.696008 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:05.697079 (+ 1071us) server_negotiation.cc:407] Connection header received
0806 01:20:05.699602 (+ 2523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:05.699610 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:05.699739 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:05.699869 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:05.701699 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.702697 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.705350 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.705804 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.705918 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:05.711185 (+ 5267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:05.711190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:05.711194 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:05.711244 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:05.716594 (+ 5350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:05.717908 (+ 1314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:05.717917 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:05.717922 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:05.718115 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:05.718681 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:05.718687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:05.718690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:05.719138 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:05.719323 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:05.721023 (+ 1700us) server_negotiation.cc:299] Negotiation successful
0806 01:20:05.721276 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":295,"threads_started":1}
I20260806 01:20:05.723395 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.695987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:46847 (local address 127.0.0.1:37434)
0806 01:20:05.696887 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.696937 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:05.699279 (+ 2342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:05.700208 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:05.700218 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:05.700791 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:05.701446 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:05.701466 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.702879 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.702886 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:05.705096 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:05.705116 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.706078 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.706088 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:05.706475 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:05.707453 (+ 978us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:05.707488 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:05.710848 (+ 3360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:05.716898 (+ 6050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:05.716913 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:05.716941 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:05.717757 (+ 816us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:05.718268 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:05.718275 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:05.718280 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:05.718531 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:05.719301 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:05.719310 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:05.719810 (+ 500us) client_negotiation.cc:769] Sending connection context
0806 01:20:05.720106 (+ 296us) client_negotiation.cc:240] Negotiation successful
0806 01:20:05.720638 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":381,"threads_started":1}
I20260806 01:20:05.725869 28255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.236.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.188@KRBTEST.COM" created.
Aug 06 01:20:05 dist-test-slave-7nm7 krb5kdc[28033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979205, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.189@KRBTEST.COM for kudu/127.25.236.189@KRBTEST.COM
I20260806 01:20:05.747316 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.726809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:38957 (local address 127.0.0.1:58316)
0806 01:20:05.726992 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.727013 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:05.727119 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:05.729217 (+ 2098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:05.729221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:05.729582 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:05.729724 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:05.729733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.732360 (+ 2627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.732369 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:05.733331 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:05.733343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.735428 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.735434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:05.735678 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:05.736453 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:05.736480 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:05.739217 (+ 2737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:05.744120 (+ 4903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:05.744126 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:05.744130 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:05.744695 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:05.745394 (+ 699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:05.745400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:05.745403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:05.745601 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:05.746474 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:05.746478 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:05.746604 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 01:20:05.746745 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 01:20:05.747010 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":45}
I20260806 01:20:05.748116 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.727055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58316 (local address 127.25.236.189:38957)
0806 01:20:05.727792 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:05.727798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:05.727826 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:20:05.728171 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:05.728184 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:05.728757 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:05.729115 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:05.729978 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.732019 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.733590 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.735155 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.735427 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:05.739494 (+ 4067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:05.739523 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:05.739547 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:05.739598 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:05.743847 (+ 4249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:05.744992 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:05.745004 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:05.745009 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:05.745193 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:05.745722 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:05.745729 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:05.745735 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:05.746314 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:05.746659 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:05.747136 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 01:20:05.747494 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":584,"thread_start_us":414,"threads_started":1}
I20260806 01:20:05.751371 28255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:20:05.752761 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.752219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:58914)
0806 01:20:05.752467 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.752609 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260806 01:20:05.753038 28189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
I20260806 01:20:05.755013 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.754639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:58926)
0806 01:20:05.754830 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.754908 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 01:20:05.755565 28255 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:43349: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.188.keytab.
Entry for principal kudu/127.25.236.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.188.keytab.
I20260806 01:20:05.783056 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.188:43349
--webserver_interface=127.25.236.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.188.keytab
--principal=kudu/127.25.236.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:34119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:46847,127.25.236.189:38957,127.25.236.188:43349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:20:05.805020 28255 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } attempt: 1
I20260806 01:20:05.806419 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.805897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:58930)
0806 01:20:05.806097 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.806207 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 01:20:05.807402 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.807053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:58942)
0806 01:20:05.807224 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.807304 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 01:20:05.807658 28255 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:43349: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
I20260806 01:20:05.813261 28149 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } attempt: 5
Aug 06 01:20:05 dist-test-slave-7nm7 krb5kdc[28033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979205, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for kudu/127.25.236.189@KRBTEST.COM
I20260806 01:20:05.830423 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.814006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:38957 (local address 127.0.0.1:58324)
0806 01:20:05.814530 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.814547 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:05.814666 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:05.815051 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:05.815054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:05.815751 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:05.816244 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:05.816259 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.817851 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.817855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:05.818929 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:05.818942 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.820120 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.820126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:05.820328 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:05.821133 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:05.821166 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:05.823826 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:05.827719 (+ 3893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:05.827724 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:05.827727 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:05.828166 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:05.828572 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:05.828576 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:05.828578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:05.828715 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:05.829345 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:05.829368 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:05.829549 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 01:20:05.829697 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 01:20:05.829984 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373}
I20260806 01:20:05.830456 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.814040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58324 (local address 127.25.236.189:38957)
0806 01:20:05.814612 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:05.814621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:05.814649 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:20:05.814772 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:05.814781 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:05.814883 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:05.815006 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:05.816422 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.817670 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.819091 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:05.819917 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:05.820077 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:05.824029 (+ 3952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:05.824035 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:05.824040 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:05.824080 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:05.827550 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:05.828304 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:05.828310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:05.828314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:05.828458 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:05.828828 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:05.828833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:05.828837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:05.829202 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:05.829348 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:05.829875 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 01:20:05.830098 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407}
I20260806 01:20:05.831944 28149 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:20:05.833578 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.832783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:58952)
0806 01:20:05.833071 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.833345 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260806 01:20:05.834961 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.834581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:58966)
0806 01:20:05.834740 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.834830 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 01:20:05.835399 28149 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:43349: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
I20260806 01:20:05.875911 28255 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } attempt: 2
I20260806 01:20:05.877197 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.876673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:58980)
0806 01:20:05.876905 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.877009 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 01:20:05.878436 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.877990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:58996)
0806 01:20:05.878163 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.878302 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260806 01:20:05.878718 28255 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:43349: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
I20260806 01:20:05.886704 28149 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } attempt: 1
I20260806 01:20:05.887984 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.887497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59006)
0806 01:20:05.887717 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.887806 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 01:20:05.889097 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.888763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59012)
0806 01:20:05.888913 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.888981 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 01:20:05.889400 28149 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:43349: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
I20260806 01:20:05.957666 28149 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } attempt: 2
I20260806 01:20:05.958962 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.958454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59024)
0806 01:20:05.958705 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.958795 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 01:20:05.960181 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.959817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59030)
0806 01:20:05.959980 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.960051 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 01:20:05.960502 28149 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:43349: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
I20260806 01:20:05.969982 28255 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } attempt: 3
I20260806 01:20:05.971424 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.970903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59032)
0806 01:20:05.971092 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.971209 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 01:20:05.972681 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:05.972156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59042)
0806 01:20:05.972472 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:05.972564 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260806 01:20:05.972889 28255 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:43349: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
I20260806 01:20:06.066749 28149 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } attempt: 3
I20260806 01:20:06.068266 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.067465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59056)
0806 01:20:06.067671 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:06.067763 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 01:20:06.069412 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.069018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59062)
0806 01:20:06.069226 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:06.069305 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260806 01:20:06.069623 28149 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:43349: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
I20260806 01:20:06.116153 28255 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } attempt: 4
I20260806 01:20:06.117426 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.116925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59078)
0806 01:20:06.117179 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:06.117272 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 01:20:06.118642 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.118290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59082)
0806 01:20:06.118471 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:06.118546 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260806 01:20:06.118904 28255 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:43349: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
W20260806 01:20:06.150089 28266 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:20:06.150521 28266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:20:06.150594 28266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:20:06.150681 28266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:20:06.158389 28266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:20:06.158497 28266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:20:06.158557 28266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:20:06.158603 28266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:20:06.158675 28266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:20:06.169004 28266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:34119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:46847,127.25.236.189:38957,127.25.236.188:43349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.188.keytab
--principal=kudu/127.25.236.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.188:43349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.236.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:20:06.170454 28266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:20:06.171759 28266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:20:06.183960 28271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:06.183915 28273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:06.184033 28275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:20:06.184975 28266 server_base.cc:1034] running on GCE node
I20260806 01:20:06.185837 28266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:20:06.188349 28266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:20:06.189628 28266 hybrid_clock.cc:648] HybridClock initialized: now 1785979206189530 us; error 69 us; skew 500 ppm
I20260806 01:20:06.189998 28266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:20:06 dist-test-slave-7nm7 krb5kdc[28033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979206, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:20:06.194063 28266 init.cc:377] Logged in from keytab as kudu/127.25.236.188@KRBTEST.COM (short username kudu)
I20260806 01:20:06.196103 28266 webserver.cc:460] Webserver started at http://127.25.236.188:40719/ using document root <none> and password file <none>
I20260806 01:20:06.196892 28266 fs_manager.cc:359] Metadata directory not provided
I20260806 01:20:06.197033 28266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:20:06.197382 28266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:20:06.200726 28266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/data/instance:
uuid: "08739b3ee8ad4652899d1243ee4240fc"
format_stamp: "Formatted at 2026-08-06 01:20:06 on dist-test-slave-7nm7"
server_key: "6562d00035dd2c9d1d02e643699d53bb"
server_key_iv: "44e15f873f3137913ea8c706ebf9e4cd"
server_key_version: "encryptionkey@0"
I20260806 01:20:06.201695 28266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/wal/instance:
uuid: "08739b3ee8ad4652899d1243ee4240fc"
format_stamp: "Formatted at 2026-08-06 01:20:06 on dist-test-slave-7nm7"
server_key: "6562d00035dd2c9d1d02e643699d53bb"
server_key_iv: "44e15f873f3137913ea8c706ebf9e4cd"
server_key_version: "encryptionkey@0"
I20260806 01:20:06.207371 28266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:20:06.211648 28284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:20:06.213203 28266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260806 01:20:06.213421 28266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/wal
uuid: "08739b3ee8ad4652899d1243ee4240fc"
format_stamp: "Formatted at 2026-08-06 01:20:06 on dist-test-slave-7nm7"
server_key: "6562d00035dd2c9d1d02e643699d53bb"
server_key_iv: "44e15f873f3137913ea8c706ebf9e4cd"
server_key_version: "encryptionkey@0"
I20260806 01:20:06.213685 28266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:20:06.240136 28266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:20:06.244783 28266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:20:06.245090 28266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:20:06.246896 28149 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } attempt: 4
I20260806 01:20:06.248060 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.247533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59092)
0806 01:20:06.247769 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:06.247885 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 01:20:06.249241 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.248812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59096)
0806 01:20:06.248985 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:06.249070 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 01:20:06.249513 28149 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:43349: Network error: Client connection negotiation failed: client connection to 127.25.236.188:43349: connect: Connection refused (error 111)
I20260806 01:20:06.265522 28266 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.188:43349
I20260806 01:20:06.265542 28354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.188:43349 every 8 connection(s)
I20260806 01:20:06.266952 28266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/data/info.pb
I20260806 01:20:06.270972 28355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:20:06.272625 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 28266
I20260806 01:20:06.272814 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978805316623-26546-0/minicluster-data/master-2/wal/instance
I20260806 01:20:06.273352 26546 external_mini_cluster.cc:1428] Setting key 4f48fa2a1ff706b73728cc6943b77991
I20260806 01:20:06.280800 28355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:20:06 dist-test-slave-7nm7 krb5kdc[28033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.188@KRBTEST.COM
Aug 06 01:20:06 dist-test-slave-7nm7 krb5kdc[28033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979206, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.188@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:20:06.298597 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.275368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59104 (local address 127.25.236.188:43349)
0806 01:20:06.276221 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.276233 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.276272 (+ 39us) server_negotiation.cc:407] Connection header received
0806 01:20:06.277826 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.277846 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.278275 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.278609 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.280424 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.281986 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.283724 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.284902 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.285095 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.289561 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:06.289569 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:06.289581 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:06.289620 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:06.293754 (+ 4134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.294699 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.294706 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.294713 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.294867 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.295357 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.295364 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.295367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.295984 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:06.296234 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:06.296766 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 01:20:06.297133 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":422,"threads_started":1}
I20260806 01:20:06.301115 28360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.283608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37448 (local address 127.25.236.190:46847)
0806 01:20:06.285015 (+ 1407us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.285021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.285039 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:20:06.285115 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.285120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.285205 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.285326 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.286926 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.287741 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.289175 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.289694 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.289792 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.294456 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:06.294464 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:06.294469 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:06.294513 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:06.297760 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.298810 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.298818 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.298824 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.299010 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.299478 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.299483 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.299486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.299841 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:06.300019 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:06.300628 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 01:20:06.300815 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1268,"thread_start_us":1123,"threads_started":1}
I20260806 01:20:06.301259 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.283493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:46847 (local address 127.0.0.1:37448)
0806 01:20:06.284305 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:06.284355 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:06.284514 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:06.285432 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:06.285440 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:06.286394 (+ 954us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:06.286756 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:06.286773 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.287907 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.287915 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:06.289007 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:06.289019 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.289901 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.289907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:06.290145 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.291097 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:06.291137 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:06.294131 (+ 2994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:06.297896 (+ 3765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:06.297906 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:06.297913 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:06.298633 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:06.299180 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:06.299184 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:06.299186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:06.299336 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:06.299995 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:06.300001 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:06.300261 (+ 260us) client_negotiation.cc:769] Sending connection context
0806 01:20:06.300486 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 01:20:06.300838 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":388,"threads_started":1}
I20260806 01:20:06.303194 28355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:20:06 dist-test-slave-7nm7 krb5kdc[28033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979206, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.188@KRBTEST.COM for kudu/127.25.236.189@KRBTEST.COM
Aug 06 01:20:06 dist-test-slave-7nm7 krb5kdc[28033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:20:06.317430 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.304067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:38957 (local address 127.0.0.1:58326)
0806 01:20:06.304251 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:06.304275 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:06.304513 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:06.304879 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:06.304884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:06.305274 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:06.305423 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:06.305431 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.306438 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.306445 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:06.307321 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:06.307333 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.308055 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.308061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:06.308273 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.308803 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:06.308820 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:06.310864 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:06.314597 (+ 3733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:06.314613 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:06.314618 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:06.315189 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:06.315586 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:06.315593 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:06.315597 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:06.315774 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:06.316297 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:06.316303 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:06.316479 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 01:20:06.316661 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 01:20:06.316967 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 01:20:06.317442 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.304020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58326 (local address 127.25.236.189:38957)
0806 01:20:06.304209 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.304215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.304438 (+ 223us) server_negotiation.cc:407] Connection header received
0806 01:20:06.304670 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.304675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.304750 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.304854 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.305570 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.306301 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.307467 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.307893 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.308033 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.311069 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:06.311075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:06.311079 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:06.311112 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:06.314464 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.315317 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.315321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.315323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.315469 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.315884 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.315887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.315889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.316166 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:06.316283 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:06.316807 (+ 524us) server_negotiation.cc:299] Negotiation successful
0806 01:20:06.317048 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:20:06.318981 28360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.302422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37452 (local address 127.25.236.190:46847)
0806 01:20:06.302616 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.302621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.302639 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:20:06.302873 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.302877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.302961 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.303094 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.304826 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.305567 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.307449 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.307874 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.308019 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.312515 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:06.312520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:06.312523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:06.312554 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:06.315918 (+ 3364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.316731 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.316736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.316739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.316888 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.317321 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.317327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.317330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.317741 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:06.318036 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:06.318394 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 01:20:06.318636 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 01:20:06.318987 28355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:20:06 dist-test-slave-7nm7 krb5kdc[28033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979206, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.188@KRBTEST.COM for kudu/127.25.236.188@KRBTEST.COM
Aug 06 01:20:06 dist-test-slave-7nm7 krb5kdc[28033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.189@KRBTEST.COM
I20260806 01:20:06.333778 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.319868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59110 (local address 127.25.236.188:43349)
0806 01:20:06.320024 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.320029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.320120 (+ 91us) server_negotiation.cc:407] Connection header received
0806 01:20:06.320265 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.320270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.320343 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.320447 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.321235 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.322007 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.323591 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.324232 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.324377 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.327803 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:06.327807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:06.327810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:06.327836 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:06.331068 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.331878 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.331882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.331884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.332004 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.332401 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.332408 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.332412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.332688 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:06.332841 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:06.333164 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 01:20:06.333385 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:20:06.333896 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.319850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59110)
0806 01:20:06.320014 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:06.320034 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:06.320159 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:06.320510 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:06.320517 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:06.320964 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:06.321121 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:06.321129 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.322137 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.322144 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:06.323438 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:06.323453 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.324426 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.324430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:06.324597 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.325257 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:06.325273 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:06.327615 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:06.331219 (+ 3604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:06.331224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:06.331227 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:06.331751 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:06.332157 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:06.332162 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:06.332166 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:06.332294 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:06.332802 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:06.332807 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:06.332937 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 01:20:06.333091 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 01:20:06.333536 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":142}
I20260806 01:20:06.337544 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.320750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58328 (local address 127.25.236.189:38957)
0806 01:20:06.320928 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.320934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.320982 (+ 48us) server_negotiation.cc:407] Connection header received
0806 01:20:06.321165 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.321180 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.321251 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.321354 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.322619 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.323646 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.325189 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.325624 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.325725 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.331261 (+ 5536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:06.331270 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:06.331274 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:06.331318 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:06.334394 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.335363 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.335369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.335372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.335530 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.335949 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.335955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.335958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.336347 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:06.336642 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:06.336875 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 01:20:06.337106 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:20:06.340651 28355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc: Bootstrap starting.
I20260806 01:20:06.343400 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:20:06.344434 28355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc: Neither blocks nor log segments found. Creating new log.
I20260806 01:20:06.345522 28355 log.cc:824] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc: Log is configured to *not* fsync() on all Append() calls
I20260806 01:20:06.348443 28355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc: No bootstrap required, opened a new log
I20260806 01:20:06.346313 28118 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1549bca01c084175848f513ec63b3fc8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37452
I20260806 01:20:06.355836 28355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } }
I20260806 01:20:06.356200 28355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08739b3ee8ad4652899d1243ee4240fc, State: Initialized, Role: FOLLOWER
I20260806 01:20:06.356642 28355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } }
I20260806 01:20:06.357667 28363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } } }
I20260806 01:20:06.358055 28363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:06.358788 28355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc [sys.catalog]: configured and running, proceeding with master startup.
W20260806 01:20:06.369005 28382 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:20:06.369426 28382 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:20:06.366966 28266 master_runner.cc:186] Error getting master registration for 127.25.236.190:46847: OK, instance_id { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" instance_seqno: 1785979205095035 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:20:06.380264 28266 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:20:06.418130 28255 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } attempt: 5
Aug 06 01:20:06 dist-test-slave-7nm7 krb5kdc[28033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979205, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.189@KRBTEST.COM for kudu/127.25.236.188@KRBTEST.COM
I20260806 01:20:06.433024 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.419011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59112 (local address 127.25.236.188:43349)
0806 01:20:06.419237 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.419245 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.419272 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:20:06.419412 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.419417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.419495 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.419603 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.420388 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.421157 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.422389 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.422899 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.423045 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.426587 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:06.426592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:06.426595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:06.426630 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:06.429893 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.430778 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.430784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.430788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.430916 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.431369 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.431374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.431378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.431716 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:06.431873 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:06.432421 (+ 548us) server_negotiation.cc:299] Negotiation successful
0806 01:20:06.432653 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:20:06.433017 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.418922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59112)
0806 01:20:06.419126 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:06.419147 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:06.419263 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:06.419638 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:06.419642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:06.420078 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:06.420235 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:06.420243 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.421305 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.421327 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:06.422240 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:06.422276 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.423306 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.423312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:06.423488 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.424130 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:06.424150 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:06.426388 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:06.430044 (+ 3656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:06.430049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:06.430054 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:06.430658 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:06.431081 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:06.431084 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:06.431086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:06.431242 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:06.431838 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:06.431843 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:06.431997 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 01:20:06.432273 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 01:20:06.432619 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 01:20:06.439294 28255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d: Bootstrap starting.
I20260806 01:20:06.443718 28255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d: Neither blocks nor log segments found. Creating new log.
I20260806 01:20:06.445159 28255 log.cc:824] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:20:06.448661 28255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d: No bootstrap required, opened a new log
I20260806 01:20:06.460285 28255 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } }
I20260806 01:20:06.460705 28255 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5e12ad72fc04103977df47eb484410d, State: Initialized, Role: FOLLOWER
I20260806 01:20:06.461289 28255 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } }
I20260806 01:20:06.462426 28387 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } } }
I20260806 01:20:06.462845 28387 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:06.463634 28255 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d [sys.catalog]: configured and running, proceeding with master startup.
W20260806 01:20:06.474215 28402 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:20:06.474464 28402 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:20:06.471269 28160 master_runner.cc:186] Error getting master registration for 127.25.236.190:46847: OK, instance_id { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" instance_seqno: 1785979205095035 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:20:06.479804 28160 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 01:20:06.497581 28292 tablet.cc:2369] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:20:06.526804 28149 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } attempt: 5
Aug 06 01:20:06 dist-test-slave-7nm7 krb5kdc[28033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979205, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for kudu/127.25.236.188@KRBTEST.COM
I20260806 01:20:06.543483 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.527600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:43349 (local address 127.0.0.1:59116)
0806 01:20:06.527867 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:06.527887 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:06.528029 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:06.528408 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:06.528412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:06.528952 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:06.529343 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:06.529364 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.530907 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.530912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:06.531853 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:06.531866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.532904 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.532910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:06.533119 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.533793 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:06.533812 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:06.536921 (+ 3109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:06.540655 (+ 3734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:06.540662 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:06.540666 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:06.541203 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:06.541625 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:06.541630 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:06.541634 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:06.541788 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:06.542424 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:06.542429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:06.542613 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 01:20:06.542762 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 01:20:06.543057 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260806 01:20:06.543483 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.527667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59116 (local address 127.25.236.188:43349)
0806 01:20:06.527862 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.527869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.528014 (+ 145us) server_negotiation.cc:407] Connection header received
0806 01:20:06.528185 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.528191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.528265 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.528381 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.529581 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.530772 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.532005 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.532762 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.532883 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.537149 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:06.537154 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:06.537157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:06.537195 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:06.540483 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.541351 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.541357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.541361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.541523 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.541905 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.541912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.541916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.542313 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:06.542455 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:06.542855 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 01:20:06.543081 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:20:06.548913 28149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8: Bootstrap starting.
I20260806 01:20:06.555855 28149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8: Neither blocks nor log segments found. Creating new log.
I20260806 01:20:06.557451 28149 log.cc:824] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8: Log is configured to *not* fsync() on all Append() calls
I20260806 01:20:06.561321 28149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8: No bootstrap required, opened a new log
I20260806 01:20:06.570115 28149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } }
I20260806 01:20:06.570601 28149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1549bca01c084175848f513ec63b3fc8, State: Initialized, Role: FOLLOWER
I20260806 01:20:06.571720 28149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } }
I20260806 01:20:06.573290 28407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } } }
I20260806 01:20:06.573974 28407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:06.575197 28149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:20:06.575381 28118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 01:20:06.575603 28118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:20:06.580475 28118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } }
W20260806 01:20:06.580814 28089 tablet.cc:2369] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:20:06.583638 28223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1549bca01c084175848f513ec63b3fc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e5e12ad72fc04103977df47eb484410d"
I20260806 01:20:06.583827 28318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1549bca01c084175848f513ec63b3fc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "08739b3ee8ad4652899d1243ee4240fc"
I20260806 01:20:06.584213 28223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:20:06.584298 28318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:20:06.588814 28318 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1549bca01c084175848f513ec63b3fc8 in term 1.
I20260806 01:20:06.588809 28223 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1549bca01c084175848f513ec63b3fc8 in term 1.
I20260806 01:20:06.589975 28084 leader_election.cc:302] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 08739b3ee8ad4652899d1243ee4240fc, 1549bca01c084175848f513ec63b3fc8, e5e12ad72fc04103977df47eb484410d; no voters:
I20260806 01:20:06.593334 28118 leader_election.cc:288] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [CANDIDATE]: Term 1 election: Requested vote from peers e5e12ad72fc04103977df47eb484410d (127.25.236.189:38957), 08739b3ee8ad4652899d1243ee4240fc (127.25.236.188:43349)
I20260806 01:20:06.601148 28407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:20:06.611706 28407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [term 1 LEADER]: Becoming Leader. State: Replica: 1549bca01c084175848f513ec63b3fc8, State: Running, Role: LEADER
I20260806 01:20:06.613456 28407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } }
I20260806 01:20:06.624150 28414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1549bca01c084175848f513ec63b3fc8. Latest consensus state: current_term: 1 leader_uuid: "1549bca01c084175848f513ec63b3fc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } } }
I20260806 01:20:06.624872 28414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [sys.catalog]: This master's current role is: LEADER
I20260806 01:20:06.630585 28422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260806 01:20:06.655318 28192 tablet.cc:2369] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:20:06.671581 28422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:20:06.697450 28360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.639211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37468 (local address 127.25.236.190:46847)
0806 01:20:06.639446 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.639455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.644178 (+ 4723us) server_negotiation.cc:407] Connection header received
0806 01:20:06.644813 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.644819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.644911 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.644990 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.653083 (+ 8093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.654262 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.658171 (+ 3909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.658998 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.659460 (+ 462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.667635 (+ 8175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:06.667641 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:06.667646 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:06.667692 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:06.671585 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.687078 (+ 15493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.687085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.687088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.687543 (+ 455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.694392 (+ 6849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.694402 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.694407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.694885 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:06.695331 (+ 446us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:06.696690 (+ 1359us) server_negotiation.cc:299] Negotiation successful
0806 01:20:06.696976 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:20:06.700941 28435 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 01:20:06.721948 28318 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc [term 1 FOLLOWER]: Refusing update from remote peer 1549bca01c084175848f513ec63b3fc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:20:06.721999 28223 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d [term 1 FOLLOWER]: Refusing update from remote peer 1549bca01c084175848f513ec63b3fc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:20:06.723254 28407 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:20:06.724012 28407 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:20:06.734175 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.642161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59118 (local address 127.25.236.188:43349)
0806 01:20:06.642456 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.642469 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.651102 (+ 8633us) server_negotiation.cc:407] Connection header received
0806 01:20:06.653744 (+ 2642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.653751 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.653838 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.653914 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.664684 (+ 10770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.666032 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.686139 (+ 20107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.687116 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.687236 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.699274 (+ 12038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:06.699283 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:06.699294 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:06.699434 (+ 140us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:06.704450 (+ 5016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.714828 (+ 10378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.714833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.714836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.715012 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.730154 (+ 15142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.730163 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.730170 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.730819 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:06.731192 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:06.733146 (+ 1954us) server_negotiation.cc:299] Negotiation successful
0806 01:20:06.733563 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 01:20:06.736202 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:06.644814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58338 (local address 127.25.236.189:38957)
0806 01:20:06.645002 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.645008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.656710 (+ 11702us) server_negotiation.cc:407] Connection header received
0806 01:20:06.657014 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.657024 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.657134 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.657222 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.678684 (+ 21462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.680135 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.688747 (+ 8612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:06.689477 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:06.689602 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:06.705628 (+ 16026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:06.705640 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:06.705645 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:06.705688 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:06.709929 (+ 4241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.714436 (+ 4507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.714445 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.714450 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.714663 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:06.718273 (+ 3610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:06.718279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:06.718282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:06.718776 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:06.719145 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:06.735282 (+ 16137us) server_negotiation.cc:299] Negotiation successful
0806 01:20:06.735665 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:20:06.751792 28363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1549bca01c084175848f513ec63b3fc8. Latest consensus state: current_term: 1 leader_uuid: "1549bca01c084175848f513ec63b3fc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } } }
I20260806 01:20:06.752170 28363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:06.752667 28436 mvcc.cc:205] Tried to move back new op lower bound from 7315370830716084224 to 7315370830310842368. Current Snapshot: MvccSnapshot[applied={T|T < 7315370830716084224}]
I20260806 01:20:06.753595 28414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1549bca01c084175848f513ec63b3fc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } } }
I20260806 01:20:06.753851 28414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [sys.catalog]: This master's current role is: LEADER
I20260806 01:20:06.755108 28422 catalog_manager.cc:1347] Generated new cluster ID: b781782f19e9420782a28aa9479760ad
I20260806 01:20:06.755203 28422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:20:06.755759 28438 mvcc.cc:205] Tried to move back new op lower bound from 7315370830716084224 to 7315370830310842368. Current Snapshot: MvccSnapshot[applied={T|T < 7315370830716084224}]
I20260806 01:20:06.759399 28363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1549bca01c084175848f513ec63b3fc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } } }
I20260806 01:20:06.759667 28363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:06.767071 28387 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1549bca01c084175848f513ec63b3fc8. Latest consensus state: current_term: 1 leader_uuid: "1549bca01c084175848f513ec63b3fc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } } }
I20260806 01:20:06.767411 28387 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:06.781651 28407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1549bca01c084175848f513ec63b3fc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } } }
I20260806 01:20:06.782045 28407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8 [sys.catalog]: This master's current role is: LEADER
I20260806 01:20:06.795543 28387 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1549bca01c084175848f513ec63b3fc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1549bca01c084175848f513ec63b3fc8" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 46847 } } peers { permanent_uuid: "e5e12ad72fc04103977df47eb484410d" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 38957 } } peers { permanent_uuid: "08739b3ee8ad4652899d1243ee4240fc" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 43349 } } }
I20260806 01:20:06.795841 28387 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:06.798174 28422 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:20:06.802562 28422 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:20:06.815909 28422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1549bca01c084175848f513ec63b3fc8: Generated new TSK 0
I20260806 01:20:06.817153 28422 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 01:20:06.962374 28360 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:06.958795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37474 (local address 127.25.236.190:46847)
0806 01:20:06.959007 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.959013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.960252 (+ 1239us) server_negotiation.cc:407] Connection header received
0806 01:20:06.960587 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.960594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.960696 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.960789 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.962092 (+ 1303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37474: BlockingRecv error: recv got EOF from 127.0.0.1:37474 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 01:20:06.964960 28358 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:06.958482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59120 (local address 127.25.236.188:43349)
0806 01:20:06.958710 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.958717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.963151 (+ 4434us) server_negotiation.cc:407] Connection header received
0806 01:20:06.963451 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.963458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.963551 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.963629 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.964688 (+ 1059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59120: BlockingRecv error: recv got EOF from 127.0.0.1:59120 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 01:20:06.966689 28262 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:06.959970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.25.236.189:38957)
0806 01:20:06.960588 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.960598 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.963399 (+ 2801us) server_negotiation.cc:407] Connection header received
0806 01:20:06.964221 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.964231 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.964599 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.964769 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.966425 (+ 1656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58344: BlockingRecv error: recv got EOF from 127.0.0.1:58344 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 01:20:06.981345 28262 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:06.977709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58358 (local address 127.25.236.189:38957)
0806 01:20:06.977906 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.977918 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.979222 (+ 1304us) server_negotiation.cc:407] Connection header received
0806 01:20:06.979506 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.979512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.979682 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.979801 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.980919 (+ 1118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58358: BlockingRecv error: recv got EOF from 127.0.0.1:58358 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 01:20:06.991288 28262 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:06.988245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58364 (local address 127.25.236.189:38957)
0806 01:20:06.988462 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.988469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:06.989617 (+ 1148us) server_negotiation.cc:407] Connection header received
0806 01:20:06.989846 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:06.989854 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:06.989949 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:06.990073 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:06.991051 (+ 978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58364: BlockingRecv error: recv got EOF from 127.0.0.1:58364 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 01:20:07.005776 28262 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:06.998405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58372 (local address 127.25.236.189:38957)
0806 01:20:06.998594 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:06.998600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:07.003438 (+ 4838us) server_negotiation.cc:407] Connection header received
0806 01:20:07.003636 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:07.003643 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:07.003746 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:07.003875 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:07.005459 (+ 1584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58372: BlockingRecv error: recv got EOF from 127.0.0.1:58372 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 01:20:07.016040 28262 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:07.013783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.25.236.189:38957)
0806 01:20:07.013960 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:07.013966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:07.014506 (+ 540us) server_negotiation.cc:407] Connection header received
0806 01:20:07.014694 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:07.014701 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:07.014801 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:07.014913 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:07.015769 (+ 856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58380: BlockingRecv error: recv got EOF from 127.0.0.1:58380 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 01:20:07.030859 28262 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:07.028396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58382 (local address 127.25.236.189:38957)
0806 01:20:07.028619 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:07.028625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:07.029173 (+ 548us) server_negotiation.cc:407] Connection header received
0806 01:20:07.029460 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:07.029467 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:07.029574 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:07.029679 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:07.030606 (+ 927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58382: BlockingRecv error: recv got EOF from 127.0.0.1:58382 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 01:20:07.052872 28262 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:07.050520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.25.236.189:38957)
0806 01:20:07.050716 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:07.050722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:07.051284 (+ 562us) server_negotiation.cc:407] Connection header received
0806 01:20:07.051482 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:07.051489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:07.051616 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:07.051732 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:07.052612 (+ 880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58396: BlockingRecv error: recv got EOF from 127.0.0.1:58396 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 01:20:07.090988 28262 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:07.088672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58404 (local address 127.25.236.189:38957)
0806 01:20:07.088931 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:07.088938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:07.089298 (+ 360us) server_negotiation.cc:407] Connection header received
0806 01:20:07.089474 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:07.089478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:07.089567 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:07.089663 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:07.090690 (+ 1027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58404: BlockingRecv error: recv got EOF from 127.0.0.1:58404 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260806 01:20:07.162101 28262 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:07.159693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58418 (local address 127.25.236.189:38957)
0806 01:20:07.159976 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:07.159983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:07.160448 (+ 465us) server_negotiation.cc:407] Connection header received
0806 01:20:07.160646 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:07.160650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:07.160766 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:07.160882 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:07.161866 (+ 984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58418: BlockingRecv error: recv got EOF from 127.0.0.1:58418 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260806 01:20:07.296109 28262 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:07.293989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58420 (local address 127.25.236.189:38957)
0806 01:20:07.294364 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:07.294372 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:07.294665 (+ 293us) server_negotiation.cc:407] Connection header received
0806 01:20:07.294921 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:07.294926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:07.294999 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:07.295111 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:07.295873 (+ 762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58420: BlockingRecv error: recv got EOF from 127.0.0.1:58420 (error 108)
Metrics: {"server-negotiator.queue_time_us":175}
I20260806 01:20:07.371701 28382 catalog_manager.cc:1259] Loaded cluster ID: b781782f19e9420782a28aa9479760ad
I20260806 01:20:07.371809 28382 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc: loading cluster ID for follower catalog manager: success
I20260806 01:20:07.374194 28382 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc: acquiring CA information for follower catalog manager: success
I20260806 01:20:07.376868 28382 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 08739b3ee8ad4652899d1243ee4240fc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:20:07.476761 28402 catalog_manager.cc:1259] Loaded cluster ID: b781782f19e9420782a28aa9479760ad
I20260806 01:20:07.476861 28402 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d: loading cluster ID for follower catalog manager: success
I20260806 01:20:07.479241 28402 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d: acquiring CA information for follower catalog manager: success
I20260806 01:20:07.480914 28402 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e5e12ad72fc04103977df47eb484410d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:20:07.563048 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:07.556751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58428 (local address 127.25.236.189:38957)
0806 01:20:07.556964 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:07.556973 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:07.557464 (+ 491us) server_negotiation.cc:407] Connection header received
0806 01:20:07.557689 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:07.557693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:07.557939 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:07.558074 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:20:07.558546 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:07.559355 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:07.560791 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:07.561176 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:07.561314 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:07.561800 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:20:07.562412 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:20:07.562574 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:07.562681 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 01:20:07.562766 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 01:20:07.576713 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:07.569988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59124 (local address 127.25.236.188:43349)
0806 01:20:07.570540 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:07.570545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:07.570563 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:20:07.570835 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:07.570840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:07.571012 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:07.571195 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:20:07.571720 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:07.572491 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:07.573971 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:07.574512 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:07.574631 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:07.575133 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:20:07.575948 (+ 815us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:20:07.576079 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:07.576275 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 01:20:07.576372 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":313,"threads_started":1}
I20260806 01:20:07.602348 28554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:07.585172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37476 (local address 127.25.236.190:46847)
0806 01:20:07.594691 (+ 9519us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:07.594696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:07.594715 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:20:07.594817 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:07.594820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:07.594990 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:07.595201 (+ 211us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:20:07.595925 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:07.597261 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:07.599169 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:07.599813 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:07.599960 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:07.600681 (+ 721us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:20:07.601146 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:20:07.601277 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:07.601859 (+ 582us) server_negotiation.cc:299] Negotiation successful
0806 01:20:07.601971 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9406,"thread_start_us":9303,"threads_started":1}
I20260806 01:20:07.603133 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:07.584971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59128 (local address 127.25.236.188:43349)
0806 01:20:07.585559 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:07.585568 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:07.596557 (+ 10989us) server_negotiation.cc:407] Connection header received
0806 01:20:07.596738 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:07.596744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:07.596855 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:07.596975 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:20:07.597622 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:07.598626 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:07.600657 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:07.601166 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:07.601553 (+ 387us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:07.602019 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:20:07.602172 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:20:07.602397 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:07.602716 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 01:20:07.602806 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251}
I20260806 01:20:07.603615 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:07.584910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.25.236.189:38957)
0806 01:20:07.585173 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:07.585197 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:07.595945 (+ 10748us) server_negotiation.cc:407] Connection header received
0806 01:20:07.597113 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:07.597121 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:07.597252 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:07.597416 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:20:07.598147 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:07.599255 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:07.601371 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:07.602049 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:07.602144 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:07.602815 (+ 671us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:20:07.602944 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:20:07.603010 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:07.603222 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 01:20:07.603306 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260806 01:20:07.615720 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 28052
I20260806 01:20:07.617374 28144 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:20:07.729393 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 28052
W20260806 01:20:07.760673 28285 connection.cc:535] server connection from 127.0.0.1:59116 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 01:20:07.761193 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 28160
I20260806 01:20:07.762478 28248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:20:07.875815 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 28160
I20260806 01:20:07.909615 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 28266
I20260806 01:20:07.911028 28345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:20:08.027091 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 28266
2026-08-06T01:20:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5293 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:20:08 dist-test-slave-7nm7 krb5kdc[28564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:20:08 dist-test-slave-7nm7 krb5kdc[28564](info): set up 2 sockets
Aug 06 01:20:08 dist-test-slave-7nm7 krb5kdc[28564](info): commencing operation
krb5kdc: starting...
W20260806 01:20:09.430120 26546 mini_kdc.cc:121] Time spent starting KDC: real 1.350s user 0.001s sys 0.064s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/test-user.keytab.
Aug 06 01:20:09 dist-test-slave-7nm7 krb5kdc[28564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:20:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:20:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
Entry for principal kudu/127.25.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab.
I20260806 01:20:09.802222 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:36463
--webserver_interface=127.25.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:36463,127.25.236.189:35283,127.25.236.188:42621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:20:10.166723 28581 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:20:10.167102 28581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:20:10.167258 28581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:20:10.167361 28581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:20:10.175100 28581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:20:10.175217 28581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:20:10.175323 28581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:20:10.175410 28581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:20:10.175505 28581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:20:10.185833 28581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:36463,127.25.236.189:35283,127.25.236.188:42621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.190.keytab
--principal=kudu/127.25.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.190:36463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:20:10.187273 28581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:20:10.188665 28581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:20:10.202257 28592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:10.202381 28587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:10.202543 28586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:20:10.203612 28581 server_base.cc:1034] running on GCE node
I20260806 01:20:10.204509 28581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:20:10.206158 28581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:20:10.207459 28581 hybrid_clock.cc:648] HybridClock initialized: now 1785979210207391 us; error 34 us; skew 500 ppm
I20260806 01:20:10.207854 28581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:20:10 dist-test-slave-7nm7 krb5kdc[28564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979210, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:20:10.211460 28581 init.cc:377] Logged in from keytab as kudu/127.25.236.190@KRBTEST.COM (short username kudu)
I20260806 01:20:10.212984 28581 webserver.cc:460] Webserver started at http://127.25.236.190:37055/ using document root <none> and password file <none>
I20260806 01:20:10.213757 28581 fs_manager.cc:359] Metadata directory not provided
I20260806 01:20:10.213897 28581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:20:10.214298 28581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:20:10.217602 28581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/data/instance:
uuid: "fd71f925acf04ecdb183d1885412afa1"
format_stamp: "Formatted at 2026-08-06 01:20:10 on dist-test-slave-7nm7"
server_key: "2db98978647860fa5b09164952bfbe84"
server_key_iv: "db61010a9b24a177d90e188ea1cdee9f"
server_key_version: "encryptionkey@0"
I20260806 01:20:10.219055 28581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/wal/instance:
uuid: "fd71f925acf04ecdb183d1885412afa1"
format_stamp: "Formatted at 2026-08-06 01:20:10 on dist-test-slave-7nm7"
server_key: "2db98978647860fa5b09164952bfbe84"
server_key_iv: "db61010a9b24a177d90e188ea1cdee9f"
server_key_version: "encryptionkey@0"
I20260806 01:20:10.226415 28581 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 01:20:10.230372 28603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:20:10.231863 28581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:20:10.232095 28581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/wal
uuid: "fd71f925acf04ecdb183d1885412afa1"
format_stamp: "Formatted at 2026-08-06 01:20:10 on dist-test-slave-7nm7"
server_key: "2db98978647860fa5b09164952bfbe84"
server_key_iv: "db61010a9b24a177d90e188ea1cdee9f"
server_key_version: "encryptionkey@0"
I20260806 01:20:10.232354 28581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:20:10.248994 28581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:20:10.253958 28581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:20:10.254427 28581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:20:10.276919 28581 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.190:36463
I20260806 01:20:10.276937 28677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.190:36463 every 8 connection(s)
I20260806 01:20:10.278494 28581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/data/info.pb
I20260806 01:20:10.279443 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 28581
I20260806 01:20:10.279668 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-0/wal/instance
I20260806 01:20:10.280537 26546 external_mini_cluster.cc:1428] Setting key 0793a3524e524ad071233c63789594ae
I20260806 01:20:10.282912 28678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:20:10.290184 28678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:20:10 dist-test-slave-7nm7 krb5kdc[28564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979210, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:20:10.316915 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.293333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:36463 (local address 127.0.0.1:49836)
0806 01:20:10.294188 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.294289 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:10.296968 (+ 2679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:10.298171 (+ 1203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:10.298177 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:10.298863 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:10.299214 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:10.299231 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.301615 (+ 2384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.301619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:10.302859 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:10.302871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.304529 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.304534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:10.304713 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:10.305504 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:10.305533 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:10.307898 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:10.312658 (+ 4760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:10.312669 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:10.312679 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:10.313106 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:10.313623 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:10.313626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:10.313628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:10.313826 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:10.314972 (+ 1146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:10.314978 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:10.315187 (+ 209us) client_negotiation.cc:769] Sending connection context
0806 01:20:10.315400 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 01:20:10.315822 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":329,"threads_started":1}
I20260806 01:20:10.317020 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.294477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49836 (local address 127.25.236.190:36463)
0806 01:20:10.295236 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:10.295243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:10.295283 (+ 40us) server_negotiation.cc:407] Connection header received
0806 01:20:10.297467 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:10.297476 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:10.297862 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:10.298082 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:10.299441 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.301410 (+ 1969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.303053 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.304247 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.304504 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:10.308173 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:10.308187 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:10.308206 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:10.308261 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:10.312496 (+ 4235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:10.313276 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:10.313288 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:10.313295 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:10.313501 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:10.313959 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:10.313967 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:10.313973 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:10.314805 (+ 832us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:10.315034 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:10.315610 (+ 576us) server_negotiation.cc:299] Negotiation successful
0806 01:20:10.316046 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":581,"thread_start_us":426,"threads_started":1}
WARNING: no policy specified for kudu/127.25.236.189@KRBTEST.COM; defaulting to no policy
I20260806 01:20:10.320363 28678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.189@KRBTEST.COM" created.
I20260806 01:20:10.322600 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.321722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:35283 (local address 127.0.0.1:39920)
0806 01:20:10.321933 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.322328 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260806 01:20:10.323011 28608 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
I20260806 01:20:10.325208 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.324451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:35283 (local address 127.0.0.1:39934)
0806 01:20:10.324789 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.325013 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260806 01:20:10.325445 28678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:35283: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.189.keytab.
Entry for principal kudu/127.25.236.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.189.keytab.
I20260806 01:20:10.362488 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.189:35283
--webserver_interface=127.25.236.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.189.keytab
--principal=kudu/127.25.236.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:36463,127.25.236.189:35283,127.25.236.188:42621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:20:10.374815 28678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } attempt: 1
I20260806 01:20:10.376037 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.375582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:35283 (local address 127.0.0.1:39948)
0806 01:20:10.375792 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.375903 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 01:20:10.377027 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.376688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:35283 (local address 127.0.0.1:39950)
0806 01:20:10.376842 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.376905 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 01:20:10.377221 28678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:35283: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
I20260806 01:20:10.445523 28678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } attempt: 2
I20260806 01:20:10.446906 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.446410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:35283 (local address 127.0.0.1:39958)
0806 01:20:10.446671 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.446769 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 01:20:10.447984 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.447601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:35283 (local address 127.0.0.1:39968)
0806 01:20:10.447770 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.447858 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 01:20:10.448169 28678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:35283: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
I20260806 01:20:10.539415 28678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } attempt: 3
I20260806 01:20:10.540608 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.540183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:35283 (local address 127.0.0.1:39972)
0806 01:20:10.540382 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.540480 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 01:20:10.541615 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.541283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:35283 (local address 127.0.0.1:39984)
0806 01:20:10.541430 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.541491 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:20:10.541785 28678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:35283: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
I20260806 01:20:10.685057 28678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } attempt: 4
I20260806 01:20:10.686535 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.685802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:35283 (local address 127.0.0.1:39998)
0806 01:20:10.686186 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.686345 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
I20260806 01:20:10.687875 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.687336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:35283 (local address 127.0.0.1:40008)
0806 01:20:10.687510 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.687622 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 01:20:10.688149 28678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.189:35283: Network error: Client connection negotiation failed: client connection to 127.25.236.189:35283: connect: Connection refused (error 111)
W20260806 01:20:10.712667 28689 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:20:10.713054 28689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:20:10.713189 28689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:20:10.713327 28689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:20:10.721834 28689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:20:10.721922 28689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:20:10.721987 28689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:20:10.722047 28689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:20:10.722116 28689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:20:10.736238 28689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:36463,127.25.236.189:35283,127.25.236.188:42621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.189.keytab
--principal=kudu/127.25.236.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.189:35283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.236.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:20:10.737650 28689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:20:10.739157 28689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:20:10.751834 28697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:10.751758 28699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:10.751866 28696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:20:10.752944 28689 server_base.cc:1034] running on GCE node
I20260806 01:20:10.753764 28689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:20:10.755329 28689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:20:10.756590 28689 hybrid_clock.cc:648] HybridClock initialized: now 1785979210756519 us; error 41 us; skew 500 ppm
I20260806 01:20:10.756953 28689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:20:10 dist-test-slave-7nm7 krb5kdc[28564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979210, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:20:10.760146 28689 init.cc:377] Logged in from keytab as kudu/127.25.236.189@KRBTEST.COM (short username kudu)
I20260806 01:20:10.761569 28689 webserver.cc:460] Webserver started at http://127.25.236.189:45815/ using document root <none> and password file <none>
I20260806 01:20:10.762338 28689 fs_manager.cc:359] Metadata directory not provided
I20260806 01:20:10.762471 28689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:20:10.762804 28689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:20:10.765815 28689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/data/instance:
uuid: "fba59c9aee0343668be489a7623031ba"
format_stamp: "Formatted at 2026-08-06 01:20:10 on dist-test-slave-7nm7"
server_key: "95d9fd09190b417363346a89c28212b5"
server_key_iv: "fb86ae7590880fb65b5deb2246ad3a97"
server_key_version: "encryptionkey@0"
I20260806 01:20:10.766666 28689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/wal/instance:
uuid: "fba59c9aee0343668be489a7623031ba"
format_stamp: "Formatted at 2026-08-06 01:20:10 on dist-test-slave-7nm7"
server_key: "95d9fd09190b417363346a89c28212b5"
server_key_iv: "fb86ae7590880fb65b5deb2246ad3a97"
server_key_version: "encryptionkey@0"
I20260806 01:20:10.771607 28689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 01:20:10.775060 28709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:20:10.776350 28689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:20:10.776549 28689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/wal
uuid: "fba59c9aee0343668be489a7623031ba"
format_stamp: "Formatted at 2026-08-06 01:20:10 on dist-test-slave-7nm7"
server_key: "95d9fd09190b417363346a89c28212b5"
server_key_iv: "fb86ae7590880fb65b5deb2246ad3a97"
server_key_version: "encryptionkey@0"
I20260806 01:20:10.776774 28689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:20:10.795933 28689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:20:10.800139 28689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:20:10.800426 28689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:20:10.819892 28689 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.189:35283
I20260806 01:20:10.819937 28784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.189:35283 every 8 connection(s)
I20260806 01:20:10.821283 28689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/data/info.pb
I20260806 01:20:10.825101 28787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:20:10.826404 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 28689
I20260806 01:20:10.826614 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-1/wal/instance
I20260806 01:20:10.827209 26546 external_mini_cluster.cc:1428] Setting key bff3d72333216b59491e40a3e8a8389f
I20260806 01:20:10.835094 28787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:20:10 dist-test-slave-7nm7 krb5kdc[28564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979210, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.189@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:20:10.860142 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.838523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49846 (local address 127.25.236.190:36463)
0806 01:20:10.839108 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:10.839116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:10.840152 (+ 1036us) server_negotiation.cc:407] Connection header received
0806 01:20:10.843106 (+ 2954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:10.843114 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:10.843220 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:10.843362 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:10.844962 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.846298 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.847864 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.848653 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.848814 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:10.852855 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:10.852860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:10.852864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:10.852901 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:10.855685 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:10.856881 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:10.856885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:10.856888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:10.857034 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:10.857707 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:10.857711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:10.857714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:10.858035 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:10.858177 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:10.859640 (+ 1463us) server_negotiation.cc:299] Negotiation successful
0806 01:20:10.859835 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":278,"threads_started":1}
I20260806 01:20:10.861819 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.839158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:36463 (local address 127.0.0.1:49846)
0806 01:20:10.839961 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.840009 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:10.842757 (+ 2748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:10.843581 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:10.843595 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:10.844157 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:10.844723 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:10.844756 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.846476 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.846480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:10.847697 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:10.847710 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.848894 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.848899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:10.849126 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:10.849881 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:10.849904 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:10.852609 (+ 2705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:10.855841 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:10.855854 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:10.855874 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:10.856736 (+ 862us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:10.857272 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:10.857280 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:10.857287 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:10.857571 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:10.858331 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:10.858343 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:10.858885 (+ 542us) client_negotiation.cc:769] Sending connection context
0806 01:20:10.859540 (+ 655us) client_negotiation.cc:240] Negotiation successful
0806 01:20:10.860159 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":312,"threads_started":1}
WARNING: no policy specified for kudu/127.25.236.188@KRBTEST.COM; defaulting to no policy
I20260806 01:20:10.864182 28787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.236.188@KRBTEST.COM" created.
Aug 06 01:20:10 dist-test-slave-7nm7 krb5kdc[28564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979210, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.189@KRBTEST.COM for kudu/127.25.236.189@KRBTEST.COM
I20260806 01:20:10.880227 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.865260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:35283 (local address 127.0.0.1:40014)
0806 01:20:10.865561 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.865587 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:10.865717 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:10.866875 (+ 1158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:10.866880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:10.867342 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:10.867476 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:10.867484 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.868876 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.868881 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:10.869711 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:10.869722 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.870594 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.870600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:10.870823 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:10.871419 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:10.871436 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:10.873380 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:10.876530 (+ 3150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:10.876535 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:10.876540 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:10.876891 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:10.877260 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:10.877265 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:10.877269 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:10.877409 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:10.878906 (+ 1497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:10.878912 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:10.879216 (+ 304us) client_negotiation.cc:769] Sending connection context
0806 01:20:10.879403 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 01:20:10.879712 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":5}
I20260806 01:20:10.880494 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.865513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40014 (local address 127.25.236.189:35283)
0806 01:20:10.866410 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:10.866416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:10.866441 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:20:10.866485 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:10.866491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:10.866726 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:10.866891 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:10.867670 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.868736 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.869848 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.870454 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.870594 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:10.873592 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:10.873602 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:10.873614 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:10.873642 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:10.876377 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:10.877021 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:10.877029 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:10.877034 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:10.877149 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:10.877511 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:10.877518 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:10.877521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:10.878684 (+ 1163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:10.879098 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:10.879616 (+ 518us) server_negotiation.cc:299] Negotiation successful
0806 01:20:10.879916 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":671,"thread_start_us":378,"threads_started":1}
I20260806 01:20:10.883095 28787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:20:10.884300 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.883749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42180)
0806 01:20:10.883965 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.884102 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 01:20:10.884598 28711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
I20260806 01:20:10.886380 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.885904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42190)
0806 01:20:10.886112 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.886213 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 01:20:10.886646 28787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:42621: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.236.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.188.keytab.
Entry for principal kudu/127.25.236.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.188.keytab.
I20260806 01:20:10.907266 26546 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVUGvte/build/asan/bin/kudu
/tmp/dist-test-taskVUGvte/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.188:42621
--webserver_interface=127.25.236.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.188.keytab
--principal=kudu/127.25.236.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.236.148:41835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.236.190:36463,127.25.236.189:35283,127.25.236.188:42621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:20:10.935994 28787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } attempt: 1
I20260806 01:20:10.937546 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.936912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42196)
0806 01:20:10.937174 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.937324 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 01:20:10.938984 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.938504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42206)
0806 01:20:10.938703 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.938816 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 01:20:10.939254 28787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:42621: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
I20260806 01:20:10.987416 28678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } attempt: 5
Aug 06 01:20:10 dist-test-slave-7nm7 krb5kdc[28564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979210, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for kudu/127.25.236.189@KRBTEST.COM
I20260806 01:20:11.001382 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.988435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40026 (local address 127.25.236.189:35283)
0806 01:20:10.988675 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:10.988682 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:10.988706 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:20:10.988774 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:10.988780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:10.988854 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:10.988964 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:10.989769 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.990560 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.991713 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.992142 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.992278 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:10.995784 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:10.995791 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:10.995795 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:10.995838 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:10.998774 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:10.999449 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:10.999461 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:10.999464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:10.999588 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:10.999974 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:10.999983 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:10.999985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.000266 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:11.000421 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:11.000812 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 01:20:11.001033 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 01:20:11.001403 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:10.988299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:35283 (local address 127.0.0.1:40026)
0806 01:20:10.988510 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:10.988535 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:10.988667 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:10.989001 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:10.989006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:10.989435 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:10.989607 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:10.989616 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.990695 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.990704 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:10.991546 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:10.991557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:10.992327 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:10.992333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:10.992544 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:10.993199 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:10.993218 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:10.995555 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:10.998927 (+ 3372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:10.998932 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:10.998936 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:10.999318 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:10.999704 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:10.999709 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:10.999713 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:10.999851 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:11.000400 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:11.000405 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:11.000562 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 01:20:11.000705 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 01:20:11.001038 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 01:20:11.002830 28678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:20:11.003862 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.003465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42210)
0806 01:20:11.003637 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.003724 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 01:20:11.004935 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.004542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42224)
0806 01:20:11.004711 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.004822 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 01:20:11.005185 28678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:42621: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
I20260806 01:20:11.007490 28787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } attempt: 2
I20260806 01:20:11.008482 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.008013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42234)
0806 01:20:11.008210 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.008309 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 01:20:11.009719 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.009188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42238)
0806 01:20:11.009437 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.009537 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260806 01:20:11.009994 28787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:42621: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
I20260806 01:20:11.056444 28678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } attempt: 1
I20260806 01:20:11.057723 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.057180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42244)
0806 01:20:11.057409 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.057523 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 01:20:11.058853 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.058485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42248)
0806 01:20:11.058639 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.058726 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 01:20:11.059089 28678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:42621: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
I20260806 01:20:11.101257 28787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } attempt: 3
I20260806 01:20:11.102511 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.101949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42250)
0806 01:20:11.102187 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.102314 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 01:20:11.103639 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.103182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42252)
0806 01:20:11.103410 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.103501 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260806 01:20:11.103868 28787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:42621: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
I20260806 01:20:11.127326 28678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } attempt: 2
I20260806 01:20:11.128562 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.128016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42256)
0806 01:20:11.128263 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.128366 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 01:20:11.129624 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.129188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42266)
0806 01:20:11.129367 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.129464 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 01:20:11.129909 28678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:42621: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
I20260806 01:20:11.236150 28678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } attempt: 3
I20260806 01:20:11.237380 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.236830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42274)
0806 01:20:11.237078 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.237184 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 01:20:11.238992 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.238501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42284)
0806 01:20:11.238769 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.238861 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260806 01:20:11.239349 28678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:42621: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
I20260806 01:20:11.247104 28787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } attempt: 4
I20260806 01:20:11.248379 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.247799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42292)
0806 01:20:11.248045 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.248200 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 01:20:11.249585 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.249196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42308)
0806 01:20:11.249360 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.249450 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 01:20:11.249903 28787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.236.188:42621: Network error: Client connection negotiation failed: client connection to 127.25.236.188:42621: connect: Connection refused (error 111)
W20260806 01:20:11.261623 28797 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:20:11.261982 28797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:20:11.262085 28797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:20:11.262276 28797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:20:11.269691 28797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:20:11.269801 28797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:20:11.269891 28797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:20:11.269972 28797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:20:11.270063 28797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:20:11.283408 28797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.236.148:41835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.236.190:36463,127.25.236.189:35283,127.25.236.188:42621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/krb5kdc/kudu_127.25.236.188.keytab
--principal=kudu/127.25.236.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.236.188:42621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.236.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:20:11.285065 28797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:20:11.286800 28797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:20:11.299428 28807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:11.299332 28808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:20:11.299362 28810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:20:11.300613 28797 server_base.cc:1034] running on GCE node
I20260806 01:20:11.301426 28797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:20:11.302927 28797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:20:11.304188 28797 hybrid_clock.cc:648] HybridClock initialized: now 1785979211304122 us; error 49 us; skew 500 ppm
I20260806 01:20:11.304598 28797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:20:11 dist-test-slave-7nm7 krb5kdc[28564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979211, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:20:11.308023 28797 init.cc:377] Logged in from keytab as kudu/127.25.236.188@KRBTEST.COM (short username kudu)
I20260806 01:20:11.309759 28797 webserver.cc:460] Webserver started at http://127.25.236.188:45655/ using document root <none> and password file <none>
I20260806 01:20:11.310539 28797 fs_manager.cc:359] Metadata directory not provided
I20260806 01:20:11.310635 28797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:20:11.310904 28797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:20:11.313853 28797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/data/instance:
uuid: "1675640e09af411cadc85ba437f8b4e6"
format_stamp: "Formatted at 2026-08-06 01:20:11 on dist-test-slave-7nm7"
server_key: "d45f6cc75e34e3ecd3740bd555c6af71"
server_key_iv: "0b9e7e45e8242fa5e3b33aa8ca4a7738"
server_key_version: "encryptionkey@0"
I20260806 01:20:11.314661 28797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/wal/instance:
uuid: "1675640e09af411cadc85ba437f8b4e6"
format_stamp: "Formatted at 2026-08-06 01:20:11 on dist-test-slave-7nm7"
server_key: "d45f6cc75e34e3ecd3740bd555c6af71"
server_key_iv: "0b9e7e45e8242fa5e3b33aa8ca4a7738"
server_key_version: "encryptionkey@0"
I20260806 01:20:11.319790 28797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 01:20:11.323891 28819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:20:11.326177 28797 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 01:20:11.326457 28797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/data,/tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/wal
uuid: "1675640e09af411cadc85ba437f8b4e6"
format_stamp: "Formatted at 2026-08-06 01:20:11 on dist-test-slave-7nm7"
server_key: "d45f6cc75e34e3ecd3740bd555c6af71"
server_key_iv: "0b9e7e45e8242fa5e3b33aa8ca4a7738"
server_key_version: "encryptionkey@0"
I20260806 01:20:11.326684 28797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:20:11.343320 28797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:20:11.347826 28797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:20:11.348067 28797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:20:11.368634 28797 rpc_server.cc:307] RPC server started. Bound to: 127.25.236.188:42621
I20260806 01:20:11.368675 28895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.236.188:42621 every 8 connection(s)
I20260806 01:20:11.370174 28797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/data/info.pb
I20260806 01:20:11.374024 28896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:20:11.374970 26546 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVUGvte/build/asan/bin/kudu as pid 28797
I20260806 01:20:11.375190 26546 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVUGvte/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978805316623-26546-0/minicluster-data/master-2/wal/instance
I20260806 01:20:11.375773 26546 external_mini_cluster.cc:1428] Setting key fe7546ed741ec9c6f95e21ff7fec855b
I20260806 01:20:11.384920 28896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:20:11 dist-test-slave-7nm7 krb5kdc[28564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.188@KRBTEST.COM
Aug 06 01:20:11 dist-test-slave-7nm7 krb5kdc[28564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979211, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.188@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:20:11.401069 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.378584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42320 (local address 127.25.236.188:42621)
0806 01:20:11.379835 (+ 1251us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.379853 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.379922 (+ 69us) server_negotiation.cc:407] Connection header received
0806 01:20:11.382558 (+ 2636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.382585 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.383055 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.383493 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.385070 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.386499 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.387979 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.389197 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.389411 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.393580 (+ 4169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:11.393595 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:11.393621 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:11.393670 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:11.396943 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.397673 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.397681 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.397688 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.397828 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.398262 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.398271 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.398276 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.398814 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:11.399121 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:11.399598 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 01:20:11.400048 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":946,"thread_start_us":475,"threads_started":1}
I20260806 01:20:11.402733 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.387566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49850 (local address 127.25.236.190:36463)
0806 01:20:11.388969 (+ 1403us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.388973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.388989 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:20:11.389046 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.389050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.389108 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.389238 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.390418 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.391119 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.392452 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.392937 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.393077 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.396787 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:11.396792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:11.396795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:11.396829 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:11.399551 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.400426 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.400434 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.400440 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.400577 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.400959 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.400964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.400968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.401280 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:11.401422 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:11.401950 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 01:20:11.402309 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1303,"thread_start_us":1189,"threads_started":1}
I20260806 01:20:11.402797 28903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.387552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.190:36463 (local address 127.0.0.1:49850)
0806 01:20:11.388382 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.388426 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:11.388569 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:11.389308 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:11.389315 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:11.389955 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:11.390209 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:11.390218 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.391280 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.391285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:11.392288 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:11.392300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.393194 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.393203 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:11.393358 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.394048 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:11.394066 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:11.396542 (+ 2476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:11.399704 (+ 3162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:11.399715 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:11.399722 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:11.400302 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:11.400701 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:11.400706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:11.400709 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:11.400845 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:11.401461 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:11.401469 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:11.401651 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 01:20:11.401840 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 01:20:11.402309 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":407,"threads_started":1}
I20260806 01:20:11.404700 28896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:20:11 dist-test-slave-7nm7 krb5kdc[28564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979211, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.188@KRBTEST.COM for kudu/127.25.236.189@KRBTEST.COM
Aug 06 01:20:11 dist-test-slave-7nm7 krb5kdc[28564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.190@KRBTEST.COM
I20260806 01:20:11.416638 28678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } attempt: 4
I20260806 01:20:11.424188 28903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.405696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.189:35283 (local address 127.0.0.1:40030)
0806 01:20:11.405967 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.406003 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:11.406127 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:11.406634 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:11.406641 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:11.407292 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:11.407552 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:11.407568 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.408966 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.408971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:11.410611 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:11.410627 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.411700 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.411704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:11.411840 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.412479 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:11.412535 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:11.415322 (+ 2787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:11.419329 (+ 4007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:11.419334 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:11.419338 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:11.419884 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:11.422297 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:11.422304 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:11.422309 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:11.422533 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:11.423179 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:11.423184 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:11.423356 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 01:20:11.423487 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 01:20:11.423805 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":1}
I20260806 01:20:11.424588 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.404362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49862 (local address 127.25.236.190:36463)
0806 01:20:11.405499 (+ 1137us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.405508 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.405538 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:20:11.405645 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.405655 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.405796 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.405955 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.408134 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.409116 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.411049 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.411790 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.411899 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.416775 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:11.416781 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:11.416785 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:11.416827 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:11.420566 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.421224 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.421229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.421233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.421656 (+ 423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.422359 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.422367 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.422371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.422758 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:11.423067 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:11.424012 (+ 945us) server_negotiation.cc:299] Negotiation successful
0806 01:20:11.424259 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":908}
I20260806 01:20:11.425034 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.405866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40030 (local address 127.25.236.189:35283)
0806 01:20:11.406050 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.406054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.406075 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:20:11.406372 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.406377 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.406447 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.406552 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.407744 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.408779 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.410805 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.411521 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.411666 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.415550 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:11.415555 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:11.415559 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:11.415591 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:11.419169 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.420020 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.420025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.420029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.420186 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.422666 (+ 2480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.422672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.422675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.423064 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:11.423229 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:11.424520 (+ 1291us) server_negotiation.cc:299] Negotiation successful
0806 01:20:11.424772 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 01:20:11 dist-test-slave-7nm7 krb5kdc[28564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979210, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.190@KRBTEST.COM for kudu/127.25.236.188@KRBTEST.COM
I20260806 01:20:11.426661 28896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:20:11.434351 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.417384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42322)
0806 01:20:11.417579 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.417607 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:11.417726 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:11.418446 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:11.418454 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:11.419006 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:11.419238 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:11.419255 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.420343 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.420349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:11.422104 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:11.422116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.423021 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.423028 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:11.423236 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.423841 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:11.423871 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:11.427208 (+ 3337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:11.431234 (+ 4026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:11.431239 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:11.431243 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:11.431834 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:11.432357 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:11.432378 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:11.432383 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:11.432578 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:11.433227 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:11.433232 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:11.433422 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 01:20:11.433592 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 01:20:11.433916 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 01:20:11.434374 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.417888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.25.236.188:42621)
0806 01:20:11.418104 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.418112 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.418132 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:20:11.418182 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.418188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.418286 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.418384 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.419413 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.420197 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.422293 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.422832 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.423553 (+ 721us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.427432 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:11.427438 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:11.427442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:11.427480 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:11.431027 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.431941 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.431949 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.431953 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.432129 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.432718 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.432723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.432726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.433081 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:11.433196 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:11.433730 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 01:20:11.433983 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 01:20:11 dist-test-slave-7nm7 krb5kdc[28564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.236.189@KRBTEST.COM
Aug 06 01:20:11 dist-test-slave-7nm7 krb5kdc[28564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979211, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.188@KRBTEST.COM for kudu/127.25.236.188@KRBTEST.COM
I20260806 01:20:11.440541 28678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1: Bootstrap starting.
I20260806 01:20:11.445101 28678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1: Neither blocks nor log segments found. Creating new log.
I20260806 01:20:11.445497 28903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.427445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42328)
0806 01:20:11.427632 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.427654 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:11.427763 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:11.429069 (+ 1306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:11.429073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:11.429628 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:11.429772 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:11.429783 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.431468 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.431473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:11.432920 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:11.432932 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.434095 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.434101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:11.434297 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.435330 (+ 1033us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:11.435346 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:11.438482 (+ 3136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:11.442606 (+ 4124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:11.442612 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:11.442616 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:11.443160 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:11.443563 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:11.443568 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:11.443571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:11.443730 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:11.444355 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:11.444360 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:11.444519 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 01:20:11.444618 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 01:20:11.444857 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 01:20:11.446376 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.426598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40040 (local address 127.25.236.189:35283)
0806 01:20:11.426753 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.426761 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.426898 (+ 137us) server_negotiation.cc:407] Connection header received
0806 01:20:11.427078 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.427084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.427164 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.427293 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.429012 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.430145 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.432064 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.432590 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.432748 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.438431 (+ 5683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:11.438438 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:11.438442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:11.438481 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:11.441980 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.443093 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.443099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.443103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.443275 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.443701 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.443707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.443711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.444115 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:11.444412 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:11.445077 (+ 665us) server_negotiation.cc:299] Negotiation successful
0806 01:20:11.445650 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 01:20:11.446750 28678 log.cc:824] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1: Log is configured to *not* fsync() on all Append() calls
I20260806 01:20:11.446839 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.427759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42328 (local address 127.25.236.188:42621)
0806 01:20:11.428784 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.428791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.428816 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:20:11.428865 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.428869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.428951 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.429052 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.429935 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.431334 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.433044 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.433936 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.435194 (+ 1258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.438689 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:11.438696 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:11.438702 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:11.438737 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:11.442438 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.443280 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.443285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.443289 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.443434 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.443858 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.443862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.443865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.444236 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:11.445976 (+ 1740us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:11.446156 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 01:20:11.446418 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":886,"thread_start_us":306,"threads_started":1}
I20260806 01:20:11.453512 28678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1: No bootstrap required, opened a new log
I20260806 01:20:11.453795 28896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6: Bootstrap starting.
I20260806 01:20:11.453835 26546 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:20:11.456928 28896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6: Neither blocks nor log segments found. Creating new log.
I20260806 01:20:11.457988 28896 log.cc:824] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6: Log is configured to *not* fsync() on all Append() calls
I20260806 01:20:11.457080 28643 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fd71f925acf04ecdb183d1885412afa1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49862
I20260806 01:20:11.460716 28896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6: No bootstrap required, opened a new log
I20260806 01:20:11.464493 28678 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } }
I20260806 01:20:11.464884 28678 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd71f925acf04ecdb183d1885412afa1, State: Initialized, Role: FOLLOWER
I20260806 01:20:11.465621 28678 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } }
I20260806 01:20:11.467060 28909 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } } }
I20260806 01:20:11.467835 28909 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:11.468648 28678 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:20:11.468755 28643 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 01:20:11.468973 28643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:20:11.471608 28896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } }
I20260806 01:20:11.472036 28896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1675640e09af411cadc85ba437f8b4e6, State: Initialized, Role: FOLLOWER
I20260806 01:20:11.472580 28896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } }
I20260806 01:20:11.472823 28643 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } }
I20260806 01:20:11.476267 28643 leader_election.cc:288] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [CANDIDATE]: Term 1 election: Requested vote from peers fba59c9aee0343668be489a7623031ba (127.25.236.189:35283), 1675640e09af411cadc85ba437f8b4e6 (127.25.236.188:42621)
I20260806 01:20:11.479115 28923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } } }
I20260806 01:20:11.479522 28923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:11.481294 28896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:20:11.481351 28860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd71f925acf04ecdb183d1885412afa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1675640e09af411cadc85ba437f8b4e6"
I20260806 01:20:11.481878 28860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:20:11.486160 28860 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd71f925acf04ecdb183d1885412afa1 in term 1.
I20260806 01:20:11.487289 28606 leader_election.cc:302] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1675640e09af411cadc85ba437f8b4e6, fd71f925acf04ecdb183d1885412afa1; no voters:
I20260806 01:20:11.487955 28909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:20:11.490303 28909 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [term 1 LEADER]: Becoming Leader. State: Replica: fd71f925acf04ecdb183d1885412afa1, State: Running, Role: LEADER
I20260806 01:20:11.491009 28909 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } }
I20260806 01:20:11.503190 28932 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd71f925acf04ecdb183d1885412afa1. Latest consensus state: current_term: 1 leader_uuid: "fd71f925acf04ecdb183d1885412afa1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } } }
I20260806 01:20:11.503630 28932 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [sys.catalog]: This master's current role is: LEADER
W20260806 01:20:11.507512 28610 tablet.cc:2369] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:20:11.509455 28955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:20:11.515928 28581 master_runner.cc:186] Error getting master registration for 127.25.236.189:35283: OK, instance_id { permanent_uuid: "fba59c9aee0343668be489a7623031ba" instance_seqno: 1785979210814365 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:20:11.537379 28581 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:20:11.539582 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.503245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.25.236.188:42621)
0806 01:20:11.503716 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.503722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.513852 (+ 10130us) server_negotiation.cc:407] Connection header received
0806 01:20:11.514298 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.514304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.514422 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.514506 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.518437 (+ 3931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.519735 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.523308 (+ 3573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.523999 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.524127 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.528820 (+ 4693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:11.528828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:11.528833 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:11.528877 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:11.533515 (+ 4638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.535493 (+ 1978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.535501 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.535507 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.535695 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.536565 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.536571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.536574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.537058 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:11.537357 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:11.538703 (+ 1346us) server_negotiation.cc:299] Negotiation successful
0806 01:20:11.539043 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354}
I20260806 01:20:11.549150 28787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } attempt: 5
I20260806 01:20:11.558326 28955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 01:20:11 dist-test-slave-7nm7 krb5kdc[28564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785979210, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.236.189@KRBTEST.COM for kudu/127.25.236.188@KRBTEST.COM
I20260806 01:20:11.571285 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.503165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49876 (local address 127.25.236.190:36463)
0806 01:20:11.503558 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.503566 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.511399 (+ 7833us) server_negotiation.cc:407] Connection header received
0806 01:20:11.526564 (+ 15165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.526573 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.526681 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.526767 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.533397 (+ 6630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.534939 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.538552 (+ 3613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.539515 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.539659 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.562114 (+ 22455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:11.562124 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:11.562129 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:11.562173 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:11.565975 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.567429 (+ 1454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.567436 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.567439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.567586 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.568346 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.568353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.568357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.568772 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:11.569051 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:11.570159 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0806 01:20:11.570600 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230}
I20260806 01:20:11.576789 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.503323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40046 (local address 127.25.236.189:35283)
0806 01:20:11.503856 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.503861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.547933 (+ 44072us) server_negotiation.cc:407] Connection header received
0806 01:20:11.548514 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.548523 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.548763 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.548861 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.551624 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.553264 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.558659 (+ 5395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.559916 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.560072 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.567269 (+ 7197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:11.567279 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:11.567287 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:11.567341 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:11.572314 (+ 4973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.573463 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.573472 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.573478 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.573689 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.574222 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.574248 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.574253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.574753 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:11.575056 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:11.575784 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 01:20:11.576188 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416}
I20260806 01:20:11.580106 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.549905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.236.188:42621 (local address 127.0.0.1:42344)
0806 01:20:11.550401 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 01:20:11.550424 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:20:11.550548 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:20:11.551630 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:20:11.551638 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:20:11.552368 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:20:11.552633 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:11.552649 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.554875 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.554883 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:11.556834 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:20:11.556855 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.558314 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.558322 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:20:11.558639 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.559295 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:20:11.559320 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:20:11.562680 (+ 3360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:20:11.567419 (+ 4739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:11.567423 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:11.567426 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:11.568528 (+ 1102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:11.576188 (+ 7660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:20:11.576193 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:20:11.576196 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:20:11.576477 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:20:11.578325 (+ 1848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:20:11.578333 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:20:11.578561 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 01:20:11.578786 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 01:20:11.579164 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334}
I20260806 01:20:11.580173 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:11.550804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42344 (local address 127.25.236.188:42621)
0806 01:20:11.551088 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.551096 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.551120 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:20:11.551188 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.551195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.551289 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.551400 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.552890 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.554628 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.557077 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:11.557887 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:11.558026 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:11.563155 (+ 5129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:20:11.563162 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:20:11.563167 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:20:11.563208 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:20:11.567260 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.569922 (+ 2662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.569928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.569931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.570219 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:20:11.576642 (+ 6423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:20:11.576652 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:20:11.576658 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:20:11.578043 (+ 1385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:20:11.578205 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:11.579031 (+ 826us) server_negotiation.cc:299] Negotiation successful
0806 01:20:11.579466 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 01:20:11.586706 28787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba: Bootstrap starting.
I20260806 01:20:11.590837 28787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba: Neither blocks nor log segments found. Creating new log.
I20260806 01:20:11.592087 28787 log.cc:824] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba: Log is configured to *not* fsync() on all Append() calls
I20260806 01:20:11.595392 28787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba: No bootstrap required, opened a new log
W20260806 01:20:11.600214 28830 tablet.cc:2369] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:20:11.604193 28860 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6 [term 1 FOLLOWER]: Refusing update from remote peer fd71f925acf04ecdb183d1885412afa1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:20:11.605196 28932 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:20:11.612382 28787 raft_consensus.cc:348] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } }
I20260806 01:20:11.613157 28787 raft_consensus.cc:729] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fba59c9aee0343668be489a7623031ba, State: Initialized, Role: FOLLOWER
I20260806 01:20:11.614094 28787 consensus_queue.cc:260] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } }
I20260806 01:20:11.616447 28969 sys_catalog.cc:455] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } } }
I20260806 01:20:11.616927 28969 sys_catalog.cc:458] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:11.620272 28750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd71f925acf04ecdb183d1885412afa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fba59c9aee0343668be489a7623031ba"
I20260806 01:20:11.620638 28747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:20:11.626142 28747 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba [term 1 FOLLOWER]: Refusing update from remote peer fd71f925acf04ecdb183d1885412afa1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:20:11.626820 28923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd71f925acf04ecdb183d1885412afa1. Latest consensus state: current_term: 1 leader_uuid: "fd71f925acf04ecdb183d1885412afa1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } } }
I20260806 01:20:11.627122 28923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:11.627597 28909 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:20:11.628928 28787 sys_catalog.cc:564] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:20:11.630442 28968 mvcc.cc:205] Tried to move back new op lower bound from 7315370850684162048 to 7315370850275434496. Current Snapshot: MvccSnapshot[applied={T|T < 7315370850684162048}]
I20260806 01:20:11.631460 28750 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd71f925acf04ecdb183d1885412afa1 in term 1.
I20260806 01:20:11.640707 28981 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 01:20:11.649834 28923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fd71f925acf04ecdb183d1885412afa1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } } }
I20260806 01:20:11.650457 28923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:11.654289 28955 catalog_manager.cc:1347] Generated new cluster ID: dc4bb2fab3184375b24c584f2187b477
I20260806 01:20:11.654420 28955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:20:11.659364 28909 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fd71f925acf04ecdb183d1885412afa1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } } }
I20260806 01:20:11.659704 28909 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [sys.catalog]: This master's current role is: LEADER
I20260806 01:20:11.659981 28797 master_runner.cc:186] Error getting master registration for 127.25.236.189:35283: OK, instance_id { permanent_uuid: "fba59c9aee0343668be489a7623031ba" instance_seqno: 1785979210814365 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:20:11.668672 28797 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:20:11.675518 28969 sys_catalog.cc:455] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd71f925acf04ecdb183d1885412afa1. Latest consensus state: current_term: 1 leader_uuid: "fd71f925acf04ecdb183d1885412afa1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } } }
I20260806 01:20:11.676021 28969 sys_catalog.cc:458] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:11.676898 28986 sys_catalog.cc:455] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fd71f925acf04ecdb183d1885412afa1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } } }
I20260806 01:20:11.677532 28986 sys_catalog.cc:458] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:20:11.678987 28909 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fd71f925acf04ecdb183d1885412afa1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd71f925acf04ecdb183d1885412afa1" member_type: VOTER last_known_addr { host: "127.25.236.190" port: 36463 } } peers { permanent_uuid: "fba59c9aee0343668be489a7623031ba" member_type: VOTER last_known_addr { host: "127.25.236.189" port: 35283 } } peers { permanent_uuid: "1675640e09af411cadc85ba437f8b4e6" member_type: VOTER last_known_addr { host: "127.25.236.188" port: 42621 } } }
I20260806 01:20:11.679349 28909 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1 [sys.catalog]: This master's current role is: LEADER
I20260806 01:20:11.686050 28955 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:20:11.689215 28955 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:20:11.712003 28998 catalog_manager.cc:1259] Loaded cluster ID: dc4bb2fab3184375b24c584f2187b477
I20260806 01:20:11.712165 28998 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba: loading cluster ID for follower catalog manager: success
I20260806 01:20:11.716302 28998 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba: acquiring CA information for follower catalog manager: success
W20260806 01:20:11.717751 28998 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260806 01:20:11.718991 28955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd71f925acf04ecdb183d1885412afa1: Generated new TSK 0
I20260806 01:20:11.721292 28955 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 01:20:11.864434 28905 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:11.855767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49890 (local address 127.25.236.190:36463)
0806 01:20:11.856014 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.856021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.860573 (+ 4552us) server_negotiation.cc:407] Connection header received
0806 01:20:11.862058 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.862064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.862440 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.862550 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.863941 (+ 1391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49890: BlockingRecv error: recv got EOF from 127.0.0.1:49890 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260806 01:20:11.865716 28793 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:11.856899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40050 (local address 127.25.236.189:35283)
0806 01:20:11.857107 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.857113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.863415 (+ 6302us) server_negotiation.cc:407] Connection header received
0806 01:20:11.863831 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.863842 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.863953 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.864037 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.865443 (+ 1406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40050: BlockingRecv error: recv got EOF from 127.0.0.1:40050 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 01:20:11.867285 28906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:11.856123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42358 (local address 127.25.236.188:42621)
0806 01:20:11.856331 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.856337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.861773 (+ 5436us) server_negotiation.cc:407] Connection header received
0806 01:20:11.865311 (+ 3538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.865318 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.865404 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.865481 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.867038 (+ 1557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42358: BlockingRecv error: recv got EOF from 127.0.0.1:42358 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 01:20:11.878605 28793 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:11.876243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40064 (local address 127.25.236.189:35283)
0806 01:20:11.876425 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.876435 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.876937 (+ 502us) server_negotiation.cc:407] Connection header received
0806 01:20:11.877175 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.877181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.877322 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.877441 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.878292 (+ 851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40064: BlockingRecv error: recv got EOF from 127.0.0.1:40064 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 01:20:11.888326 28793 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:11.884712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40078 (local address 127.25.236.189:35283)
0806 01:20:11.884901 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.884907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.886849 (+ 1942us) server_negotiation.cc:407] Connection header received
0806 01:20:11.887045 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.887051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.887140 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.887257 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.888094 (+ 837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40078: BlockingRecv error: recv got EOF from 127.0.0.1:40078 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 01:20:11.901350 28793 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:11.899061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40090 (local address 127.25.236.189:35283)
0806 01:20:11.899258 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.899265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.899825 (+ 560us) server_negotiation.cc:407] Connection header received
0806 01:20:11.900019 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.900026 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.900145 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.900265 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.901083 (+ 818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40090: BlockingRecv error: recv got EOF from 127.0.0.1:40090 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 01:20:11.910652 28793 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:11.908428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40106 (local address 127.25.236.189:35283)
0806 01:20:11.908646 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.908652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.909178 (+ 526us) server_negotiation.cc:407] Connection header received
0806 01:20:11.909359 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.909365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.909470 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.909581 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.910415 (+ 834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40106: BlockingRecv error: recv got EOF from 127.0.0.1:40106 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260806 01:20:11.924937 28793 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:11.922166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40110 (local address 127.25.236.189:35283)
0806 01:20:11.922392 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.922398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.923026 (+ 628us) server_negotiation.cc:407] Connection header received
0806 01:20:11.923444 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.923453 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.923667 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.923800 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.924665 (+ 865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40110: BlockingRecv error: recv got EOF from 127.0.0.1:40110 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260806 01:20:11.947892 28793 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:11.944867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40118 (local address 127.25.236.189:35283)
0806 01:20:11.945082 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.945088 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.945691 (+ 603us) server_negotiation.cc:407] Connection header received
0806 01:20:11.945915 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.945921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.946038 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.946174 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.947585 (+ 1411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40118: BlockingRecv error: recv got EOF from 127.0.0.1:40118 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260806 01:20:11.987303 28793 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:11.984307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40132 (local address 127.25.236.189:35283)
0806 01:20:11.984511 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:11.984518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:11.985557 (+ 1039us) server_negotiation.cc:407] Connection header received
0806 01:20:11.985783 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:11.985789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:11.985903 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:11.986031 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:11.987009 (+ 978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40132: BlockingRecv error: recv got EOF from 127.0.0.1:40132 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 01:20:12.066773 28793 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:12.064198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40134 (local address 127.25.236.189:35283)
0806 01:20:12.064439 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:12.064446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:12.065096 (+ 650us) server_negotiation.cc:407] Connection header received
0806 01:20:12.065360 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:12.065366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:12.065459 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:12.065560 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:12.066539 (+ 979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40134: BlockingRecv error: recv got EOF from 127.0.0.1:40134 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260806 01:20:12.200121 28793 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:12.198374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40148 (local address 127.25.236.189:35283)
0806 01:20:12.198594 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:12.198603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:12.198917 (+ 314us) server_negotiation.cc:407] Connection header received
0806 01:20:12.199079 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:12.199084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:12.199160 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:12.199267 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:12.199916 (+ 649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40148: BlockingRecv error: recv got EOF from 127.0.0.1:40148 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 01:20:12.467034 28793 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:20:12.465065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.25.236.189:35283)
0806 01:20:12.465307 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:12.465314 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:12.465546 (+ 232us) server_negotiation.cc:407] Connection header received
0806 01:20:12.465792 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:12.465804 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:12.465877 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:12.465988 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:20:12.466856 (+ 868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40156: BlockingRecv error: recv got EOF from 127.0.0.1:40156 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 01:20:12.646730 28981 catalog_manager.cc:1259] Loaded cluster ID: dc4bb2fab3184375b24c584f2187b477
I20260806 01:20:12.646878 28981 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6: loading cluster ID for follower catalog manager: success
I20260806 01:20:12.649564 28981 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6: acquiring CA information for follower catalog manager: success
I20260806 01:20:12.651460 28981 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1675640e09af411cadc85ba437f8b4e6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:20:12.720526 28998 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fba59c9aee0343668be489a7623031ba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:20:12.991781 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:12.984177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40164 (local address 127.25.236.189:35283)
0806 01:20:12.984929 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:12.984934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:12.984952 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:20:12.985016 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:12.985020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:12.985169 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:12.985279 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:20:12.985775 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:12.986648 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:12.988929 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:12.989550 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:12.989673 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:12.990393 (+ 720us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:20:12.990908 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:20:12.991044 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:12.991362 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 01:20:12.991481 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":477,"threads_started":1}
I20260806 01:20:13.019117 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:12.998780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42364 (local address 127.25.236.188:42621)
0806 01:20:13.011912 (+ 13132us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:13.011917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:13.011941 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:20:13.012039 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:13.012043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:13.012553 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:13.012975 (+ 422us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:20:13.013601 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:13.014509 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:13.016409 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:13.016861 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:13.016961 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:13.017733 (+ 772us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:20:13.018356 (+ 623us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:20:13.018513 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:13.018642 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 01:20:13.018712 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13006,"thread_start_us":12810,"threads_started":1}
I20260806 01:20:13.042749 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:13.032217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42374 (local address 127.25.236.188:42621)
0806 01:20:13.032425 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:13.032433 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:13.037030 (+ 4597us) server_negotiation.cc:407] Connection header received
0806 01:20:13.037303 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:13.037308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:13.037386 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:13.037487 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:20:13.038043 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:13.038962 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:13.040744 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:13.041161 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:13.041295 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:13.041918 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:20:13.042037 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:20:13.042142 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:13.042367 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 01:20:13.042449 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 01:20:13.044884 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:13.026360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40166 (local address 127.25.236.189:35283)
0806 01:20:13.026592 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:13.026602 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:13.037293 (+ 10691us) server_negotiation.cc:407] Connection header received
0806 01:20:13.037551 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:13.037556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:13.037641 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:13.037705 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:20:13.038799 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:13.039606 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:13.042325 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:13.043063 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:13.043265 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:13.044074 (+ 809us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:20:13.044197 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:20:13.044280 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:13.044493 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 01:20:13.044575 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 01:20:13.045096 29116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:20:13.025971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49904 (local address 127.25.236.190:36463)
0806 01:20:13.035815 (+ 9844us) server_negotiation.cc:206] Beginning negotiation
0806 01:20:13.035824 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:20:13.036677 (+ 853us) server_negotiation.cc:407] Connection header received
0806 01:20:13.037149 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:20:13.037159 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:20:13.037394 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:20:13.037546 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:20:13.038768 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:13.039820 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:13.042352 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:20:13.042739 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:20:13.043017 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:20:13.043677 (+ 660us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:20:13.044143 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:20:13.044253 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 01:20:13.044674 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 01:20:13.044766 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9659,"thread_start_us":9860,"threads_started":1}
I20260806 01:20:13.054479 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 28581
I20260806 01:20:13.055883 28673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:20:13.173341 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 28581
I20260806 01:20:13.209041 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 28689
I20260806 01:20:13.210392 28776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:20:13.320472 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 28689
I20260806 01:20:13.352054 26546 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskVUGvte/build/asan/bin/kudu pid 28797
I20260806 01:20:13.353210 28890 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:20:13.460083 26546 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVUGvte/build/asan/bin/kudu with pid 28797
2026-08-06T01:20:13Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5428 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10721 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (408038 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST